New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add tutorial and sample code for TensorFlow plugin development #352

Merged

theadactyl merged 30 commits into tensorflow:master from jzhoulon:plugin_tutorial

Oct 19, 2022

Contributor

jzhoulon commented Feb 1, 2021

Add tutorial and code sample for TensorFlow plugin development

jzhoulon requested review from ematejska, ewilderj and theadactyl as code owners

February 1, 2021 11:08

google-cla bot added the cla: yes label


          Add tutorial and sample code for TensorFlow plugin development

260049d

Contributor

theadactyl commented Feb 1, 2021

@mihaimaruseac @penpornk can you review?

Member

penpornk commented Feb 1, 2021

@theadactyl Yes, I will review it soon.

penpornk suggested changes

View reviewed changes

Member

penpornk left a comment

Thank you very much for this! :)
I think we should make the demo code download necessary third_party files from TensorFlow instead of storing them here.
Also, there is a stray eigen.tar.gz file in //sample/third_party/eigen3. Please remove.

rfcs/20200624-pluggable-device-for-tensorflow/sample/README.md Outdated

+              # TensorFlow Plugin demo
+              This sample is a simple demo shows how to implement, build, install and run a TensorFlow plugin.
+              ##Supported OS

Member

penpornk Feb 2, 2021

Nit: Please add a space after ## or the text won't be rendered as header. Same for other places in this file.

Suggested change

      
            ##Supported OS
          
            ## Supported OS

Contributor Author

jzhoulon Feb 3, 2021

done

rfcs/20200624-pluggable-device-for-tensorflow/sample/README.md Outdated

+              * Python (version 3.6 and above)
+              ##Build and Run
+              (This dependes on ([PluggableDevice] PluggableDevice mechanism implementation)[https://github.com/tensorflow/tensorflow/pull/45784] PR merged)

Member

penpornk Feb 2, 2021

Nit: The link was not formatted correctly.
Also, would we need PR #43611 (device alias) as well?

Suggested change

      
            (This dependes on ([PluggableDevice] PluggableDevice mechanism implementation)[https://github.com/tensorflow/tensorflow/pull/45784] PR merged)
          
            This demo depends on the following ongoing PRs: 
          
            * [\[PluggableDevice\] PluggableDevice mechanism implementation](https://github.com/tensorflow/tensorflow/pull/45784).
          
            * [\[PluggableDevice\] Add device alias support](https://github.com/tensorflow/tensorflow/pull/43611)

Contributor Author

jzhoulon Feb 3, 2021

the device alias PR is only needed when plugin author want to register their device as a "GPU" device(legacy gpu support)

rfcs/20200624-pluggable-device-for-tensorflow/sample/README.md Outdated


		##Prerequisites

		* Bazel (version 3.1 and above)

Member

penpornk Feb 2, 2021

Let's add a link for Bazel. :)

Suggested change

      
            * Bazel (version 3.1 and above)
          
            * [Bazel](https://docs.bazel.build/versions/master/install-ubuntu.html) (version 3.1 and above)

Contributor Author

jzhoulon Feb 3, 2021

Done

rfcs/20200624-pluggable-device-for-tensorflow/sample/README.md Outdated

+              ###Linux
+. Run the following commands to install a tf-nightly.
+              ```
+              >>pip install tf-nightly

Member

penpornk Feb 2, 2021

Nit: Could you please add a space after >>? Same for other places in this file.

Suggested change

      
            >>pip install tf-nightly
          
            >> pip install tf-nightly

Contributor Author

jzhoulon Feb 3, 2021

Done

rfcs/20200624-pluggable-device-for-tensorflow/sample/README.md Outdated

+              (This dependes on ([PluggableDevice] PluggableDevice mechanism implementation)[https://github.com/tensorflow/tensorflow/pull/45784] PR merged)
+              ###Linux
+. Run the following commands to install a tf-nightly.

Member

penpornk Feb 2, 2021

Nit:

Suggested change

      
            1. Run the following commands to install a tf-nightly.
          
            1. Run the following command to install `tf-nightly`.

Contributor Author

jzhoulon Feb 3, 2021

Done

rfcs/20200624-pluggable-device-for-tensorflow/sample/README.md Outdated

+              Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native -Wno-sign-compare]:
+              ```
+. Then built it with

Member

penpornk Feb 2, 2021

Nit:

Suggested change

      
            3. Then built it with
          
            3. Build the plug-in with

Contributor Author

jzhoulon Feb 3, 2021

Done

rfcs/20200624-pluggable-device-for-tensorflow/sample/configure.py Outdated

+                    Raise the error to avoid infinitely looping.
+                """
+                if not question:
+                  question = 'Do you wish to build TensorFlow with %s support?' % query_item

Member

penpornk Feb 2, 2021

Nit:

Suggested change

      
                question = 'Do you wish to build TensorFlow with %s support?' % query_item
          
                question = 'Do you wish to build TensorFlow plug-in with %s support?' % query_item

Contributor Author

jzhoulon Feb 3, 2021

Done

rfcs/20200624-pluggable-device-for-tensorflow/sample/configure.py Outdated

+                if not question:
+                  question = 'Do you wish to build TensorFlow with %s support?' % query_item
+                if not yes_reply:
+                  yes_reply = '%s support will be enabled for TensorFlow.' % query_item

Member

penpornk Feb 2, 2021

Nit:

Suggested change

      
                yes_reply = '%s support will be enabled for TensorFlow.' % query_item
          
                yes_reply = '%s support will be enabled for TensorFlow plug-in.' % query_item

Contributor Author

jzhoulon Feb 3, 2021

Done

rfcs/20200624-pluggable-device-for-tensorflow/sample/eigen.BUILD Outdated

		@@ -0,0 +1,78 @@
		# Description:
		# Eigen is a C++ template library for linear algebra: vectors,

Member

penpornk Feb 2, 2021

This file and the folder //sample/eigen3 seems to be redundant to the ones in //sample/third_party. Please delete them if they are indeed duplicates.

Contributor Author

jzhoulon Feb 3, 2021

done, removed it

rfcs/20200624-pluggable-device-for-tensorflow/sample/third_party/eigen.BUILD

		@@ -0,0 +1,79 @@
		# Description:
		# Eigen is a C++ template library for linear algebra: vectors,

Member

penpornk Feb 2, 2021

Since the whole third_party/ folder is quite big, can we just download that folder from TensorFlow and keep only what we need? For example, we could add this to build.sh before the bazel build command.

COMMIT=2eea5090c05a1cb99e631aecefd107597afd227b
wget https://github.com/tensorflow/tensorflow/archive/${COMMIT}.tar.gz -O tensorflow.tar.gz
tar -xvzf tensorflow.tar.gz tensorflow-${COMMIT}/third_party/ 
mv tensorflow-${COMMIT}/third_party/eigen3 third_party/
 ...  # Move other files.
rm -rf tensorflow-${COMMIT} tensorflow.tar.gz

This will make the demo code more compact.

Contributor Author

jzhoulon Feb 3, 2021

The demo only depends on eigen_archive and com_google_absl in third-party currently, the version is the same as proper, do we need to download the whole tensorflow package(seems too big).

jzhoulon added 9 commits

February 2, 2021 16:30


          fix conv demo code

8c292c7


          remove useless file

482cbd4


          remove useless code

8aa3d42


          modify the format

7867d60


          Fix README format

cb64115


          fix README

99106ff


          Fix README

ee29b3c


          add plug-in run example

a5cdabc


          fix README format for plug-in

73b0c24

penpornk mentioned this pull request

RFC: TensorFlow on DirectML #243

Merged

PatriceVignola commented May 25, 2021

Hi,

I've been following the tutorial and it's pretty useful, but I have one question: what should be the name of the plugin package? In TensorFlow I see that it's explicitly looking for a python package named tensorflow_plugins, but wouldn't that mean that 2 plugins can't be used simultaneously? Is there a way to register a pluggable device with a different name?

Let me know if this isn't the right place to ask this question.

Thanks!

Member

penpornk commented May 26, 2021 •

edited

Loading

Hi @PatriceVignola,

Yes, here is the right place to ask.

TensorFlow doesn't exactly look for a package named tensorflow_plugins, it's just going to look for shared libraries (.so files for linux) in a folder named tensorflow-plugins. That folder can have multiple shared libraries from multiple plugin packages, e.g., apu.so, bpu.so, dpu.so, etc.

I think the current pip-package generation script in this tutorial puts more than just the shared library file into the tensorflow-plugins folder when the package is installed. So it might need some adjustments.

Contributor Author

jzhoulon commented May 26, 2021

@PatriceVignola each package can have its own package name,the only requirement is the pluggable device library should be copied to tensorflow-plugins folder, I will update the tutoral example code. Thanks

PatriceVignola commented May 26, 2021

Thanks! That makes sense.

quintinwang5 reviewed

View reviewed changes

...20200624-pluggable-device-for-tensorflow/sample/tensorflow_plugin/tools/pip_package/setup.py Outdated Show resolved Hide resolved

jzhoulon and others added 2 commits

May 25, 2021 22:08


          Update rfcs/20200624-pluggable-device-for-tensorflow/sample/tensorflo…

b3a4cec

…w_plugin/tools/pip_package/setup.py

Co-authored-by: quintinwang5 <[email protected]>


          make package install two directory

626bb31

Contributor Author

jzhoulon commented May 28, 2021

@penpornk we have update the graph optimization C API part, including sample code. Thanks.

jzhoulon added 3 commits

May 28, 2021 15:47


          update graph


          update graph flow

be5e24b


          fix bug

0bd8a73

PatriceVignola commented May 29, 2021

What would be the best way to get the equivalent of libtensorflow_framework.so in Windows without building TensorFlow from source (if any)? It's not included as part of the python package and there doesn't seem to be Windows nightly builds for the C API library.

Thank you!

jzhoulon added 14 commits

October 21, 2021 09:48


          add profiler sample code

d888a7d


          add profiler test case

40bf8d4


          update README

8a776a7


          update README

028800a


          update README

a7e30cc


          remove useless file

a835c14


          update image

ea1d520


          update tutorial

fc90602


          update image

cf541dd


          updata profiler tutorial part

1c78e90


          upload XSpace img

4433d5c


          update tutorial


          update profiler

fe26181


          add profiler example

673cda8

xiaoshiyilang111 reviewed

View reviewed changes

xiaoshiyilang111 left a comment

I try to use the sample code with
tf 2.5.0 cpu version， but error happened!
I compiled a library named mydevice_tf.so using gcc, and copy it into python/lib/site-packages/tensorflow-2.5.0-py3.6-linux-x86_64.egg/tensorflow/python/tensorflow-plugins.
when I run python relu.py, I got this:
Unkown attribute: 'MY_Devices' in '/MY_DEVICE:0'

Contributor

ematejska commented Dec 9, 2021

@penpornk Have all your changes been addressed? If so, could you re-review?

dsseng commented Jul 22, 2022 •

edited

Loading

I have (partially) ported this sample code to Rust. Which way should I better post it? Could I publish that template in my repository or submit it to get included into this PR?

For now it only includes unsafe port of most plugin registration features except profiling and optimization.

Member

penpornk commented Jul 22, 2022

@sh7dm Thank you for your contribution! We will try to merge this PR soon. Then you can open a new PR to add new changes. How does this sound? :)

dsseng commented Jul 22, 2022

Great! May I push stuff I have into an own repo at least for now (of course linking to this PR as the initial source)?

dsseng commented Jul 23, 2022

Let it be here for now.

dsseng added a commit to dsseng/rust-tf-pluggabledevice that referenced this pull request


          Port basic plugin to unsafe Rust code

32fbe49

Code ported from tensorflow/community#352 example

dsseng commented Jul 23, 2022

I'm now working on extending my Rust port with stuff like device-bound kernels (accessing device from kernel).

dsseng reviewed

View reviewed changes

rfcs/20200624-pluggable-device-for-tensorflow/tutorial.md

    
              1. We need to include those C API header files provided by Core TensorFlow.

              2. The built plugin library needs to add dependency to `_pywrap_tensorflow_internal.so`, which is built by Core TensorFlow. `_pywrap_tensorflow_internal.so` contains those C API implementations. If you don’t add this dependency, it will report an "undefined symbol" error when loading the plugin library.

dsseng Jul 24, 2022

Working on my Rust port I found that linking to libtensorflow_framework.so.2 works fine as well, but does not add a dependency on Python, which is not necessary and limits the possibilities. I changed those libraries when I found that tests do not build because of linking dependency on Python's C APIs, but with C-native lib everything is fine.

See dsseng/rust-tf-pluggabledevice@d1d812c

Contributor

ematejska commented Jul 25, 2022

@penpornk Once you approve, we'll get this merged.

penpornk suggested changes

View reviewed changes

Member

penpornk left a comment

@jzhoulon I sincerely agologize again for the delay. :(

Would you mind making some more changes to the code? Even though they are tutorial code, we should still add the license message to the source files.

...pluggable-device-for-tensorflow/sample/tensorflow_plugin/src/device/cpu/cpu_device_plugin.cc

		@@ -0,0 +1,279 @@
		#include "tensorflow_plugin/src/device/cpu/cpu_device_plugin.h"

Member

penpornk Oct 12, 2022

Could you please add the usual license message to every C++ source/header files?

Contributor Author

jzhoulon Oct 17, 2022

done

rfcs/20200624-pluggable-device-for-tensorflow/sample/tensorflow_plugin/python/test.py

		@@ -0,0 +1,6 @@
		"""This is a module for dummy test."""

Member

penpornk Oct 12, 2022

Could you please add the usual license message to all python source files?

Contributor Author

jzhoulon Oct 17, 2022

done


          Add license for C++ and python files

penpornk approved these changes

View reviewed changes

Member

penpornk left a comment

Thank you very much for the tutorial and I'm so sorry again for the delay!

njzjz mentioned this pull request

[Feature Request] Migrate from TF C++ API to TF C API deepmodeling/deepmd-kit#2008

Closed

theadactyl approved these changes

View reviewed changes

theadactyl merged commit 22d927e into tensorflow:master

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

quintinwang5 quintinwang5 left review comments

xiaoshiyilang111 xiaoshiyilang111 left review comments

dsseng dsseng left review comments

penpornk penpornk approved these changes

theadactyl theadactyl approved these changes

ematejska Awaiting requested review from ematejska

ewilderj Awaiting requested review from ewilderj

Labels