From 0d20478f6859ebfeed1a9b0e32058d3a27ec78cc Mon Sep 17 00:00:00 2001 From: Hendrik Borras Date: Thu, 26 Aug 2021 19:36:05 +0100 Subject: [PATCH] Added support for cost estimation for upsampling (#37) * Update AUTHORS.rst * Create python-publish.yml * Add ZCU111 board to part map (#32) * Update AUTHORS.rst * GHA for PyPI, sdist only (#35) * finn-base v0.0.2 (#34) * Modified set_nodeattr to allow using it on repeated fields (#18) * [base]: changed how the floats, ints, strings, tensors, graphs and sparse_tensors field of AttributeProto is set. * [Core] restrict attributes to tested types Co-authored-by: Yaman Umuroglu * Support for non-square input images and kernels for im2col node (#20) * [im2col.py]: support for non-square input images and kernels, [test_im2col]: added/modified several test cases for (non-)square images and kernels * [im2col]: support for non-square input images and kernels, [test_im2col]: added/modified several test cases for (non-)square images and kernels * [test_general_transformation]: changed kernel_size attribute to list instead of integer as required by im2col node * [base]: changed how the "ints" field of AttributeProto set. [test_general_transformation]: changed the type of the kernel_size attribute to list of integers * removed unused import * [base]: added support for writing repeated fields in AttributeProto * minor style changes * [im2col, test_im2col]: added support for non-equal padding * [lower_convs_to_matmul]: added support for non-square input images and kernels and non-equal padding. [test_conv_lowering]: added/modified test cases for non-equal padding, depthwise convolution and 'standard' convolution. * [test_conv_lowering]: included 1D depthwise and regular convolutions in tests * Revert "[test_conv_lowering]: included 1D depthwise and regular convolutions in tests" This reverts commit 3ff449c42d709e640ca904c41a241bb94fc9e335 * Revert "[lower_convs_to_matmul]: added support for non-square input images and kernels and non-equal padding." This reverts commit 15e34ed8d07d4a55996f162bd3bd1aa24b33c3ac. * Revert "[im2col, test_im2col]: added support for non-equal padding" This reverts commit c524020ee8a7b363eb0c30d70cf21332e9c73678. * [im2col]: changed how a square kernel is instantiated. [lower_convs_to_matmul]: changed how the kernel size attribute is read (based on how a square kernel is instantiated). * [im2col]: minor change in style. * [Im2Col] style fixes and comments Co-authored-by: Yaman Umuroglu * Update AUTHORS.rst * Support for non-square input images and kernels in LowerConvsToMatMul transformation (#16) * [im2col.py]: support for non-square input images and kernels, [test_im2col]: added/modified several test cases for (non-)square images and kernels * [im2col]: support for non-square input images and kernels, [test_im2col]: added/modified several test cases for (non-)square images and kernels * [test_general_transformation]: changed kernel_size attribute to list instead of integer as required by im2col node * [base]: changed how the "ints" field of AttributeProto set. [test_general_transformation]: changed the type of the kernel_size attribute to list of integers * removed unused import * [base]: added support for writing repeated fields in AttributeProto * minor style changes * [im2col, test_im2col]: added support for non-equal padding * [lower_convs_to_matmul]: added support for non-square input images and kernels and non-equal padding. [test_conv_lowering]: added/modified test cases for non-equal padding, depthwise convolution and 'standard' convolution. * [test_conv_lowering]: included 1D depthwise and regular convolutions in tests * Revert "[test_conv_lowering]: included 1D depthwise and regular convolutions in tests" This reverts commit 3ff449c42d709e640ca904c41a241bb94fc9e335 * Revert "[lower_convs_to_matmul]: added support for non-square input images and kernels and non-equal padding." This reverts commit 15e34ed8d07d4a55996f162bd3bd1aa24b33c3ac. * Revert "[im2col, test_im2col]: added support for non-equal padding" This reverts commit c524020ee8a7b363eb0c30d70cf21332e9c73678. * [im2col] function compute_conv_output_dim can now be called in case non-equal and equal padding is assumed. [test_im2col] changed function call to compute_conv_output_dim. [test_conv_lowering] changed function call to compute_conv_output_dim. [lower_convs_to_matmul] removed old assertion. * [im2col]: changed how a square kernel is instantiated. [lower_convs_to_matmul]: changed how the kernel size attribute is read (based on how a square kernel is instantiated). * [im2col]: changed how a square kernel is instantiated. [lower_convs_to_matmul]: changed how the kernel size attribute is read (based on how a square kernel is instantiated). * [im2col]: minor change in style. * [test_conv_lowering]: minor fix for test case depthwise and regular convolutions * Support for non-square input images and kernels for im2col node (#20) * [im2col.py]: support for non-square input images and kernels, [test_im2col]: added/modified several test cases for (non-)square images and kernels * [im2col]: support for non-square input images and kernels, [test_im2col]: added/modified several test cases for (non-)square images and kernels * [test_general_transformation]: changed kernel_size attribute to list instead of integer as required by im2col node * [base]: changed how the "ints" field of AttributeProto set. [test_general_transformation]: changed the type of the kernel_size attribute to list of integers * removed unused import * [base]: added support for writing repeated fields in AttributeProto * minor style changes * [im2col, test_im2col]: added support for non-equal padding * [lower_convs_to_matmul]: added support for non-square input images and kernels and non-equal padding. [test_conv_lowering]: added/modified test cases for non-equal padding, depthwise convolution and 'standard' convolution. * [test_conv_lowering]: included 1D depthwise and regular convolutions in tests * Revert "[test_conv_lowering]: included 1D depthwise and regular convolutions in tests" This reverts commit 3ff449c42d709e640ca904c41a241bb94fc9e335 * Revert "[lower_convs_to_matmul]: added support for non-square input images and kernels and non-equal padding." This reverts commit 15e34ed8d07d4a55996f162bd3bd1aa24b33c3ac. * Revert "[im2col, test_im2col]: added support for non-equal padding" This reverts commit c524020ee8a7b363eb0c30d70cf21332e9c73678. * [im2col]: changed how a square kernel is instantiated. [lower_convs_to_matmul]: changed how the kernel size attribute is read (based on how a square kernel is instantiated). * [im2col]: minor change in style. * [Im2Col] style fixes and comments Co-authored-by: Yaman Umuroglu * Update AUTHORS.rst Co-authored-by: Yaman Umuroglu * Added support for dilation value = 2 for 1D and 2D images/kernels (#17) * [im2col.py]: support for non-square input images and kernels, [test_im2col]: added/modified several test cases for (non-)square images and kernels * [im2col]: support for non-square input images and kernels, [test_im2col]: added/modified several test cases for (non-)square images and kernels * [test_general_transformation]: changed kernel_size attribute to list instead of integer as required by im2col node * [base]: changed how the "ints" field of AttributeProto set. [test_general_transformation]: changed the type of the kernel_size attribute to list of integers * removed unused import * [base]: added support for writing repeated fields in AttributeProto * minor style changes * [im2col, test_im2col]: added support for non-equal padding * [lower_convs_to_matmul]: added support for non-square input images and kernels and non-equal padding. [test_conv_lowering]: added/modified test cases for non-equal padding, depthwise convolution and 'standard' convolution. * [test_conv_lowering]: included 1D depthwise and regular convolutions in tests * Revert "[test_conv_lowering]: included 1D depthwise and regular convolutions in tests" This reverts commit 3ff449c42d709e640ca904c41a241bb94fc9e335 * Revert "[lower_convs_to_matmul]: added support for non-square input images and kernels and non-equal padding." This reverts commit 15e34ed8d07d4a55996f162bd3bd1aa24b33c3ac. * Revert "[im2col, test_im2col]: added support for non-equal padding" This reverts commit c524020ee8a7b363eb0c30d70cf21332e9c73678. * [im2col] added support for dilations = 2 for 2D and 1D images. Dilation value must be equal along each axis. [test_im2col] added several test cases in case dilations = 2 (a.o. cases where image and kernel are 2D and 1D, with and without padding, and with stride = 2). [lower_convs_to_matmul] added support for dilation value. Dilation value must be equal along each axis. [test_conv_lowering] added test case for dilations = 2 for 2D and 1D images and kernels, with and without padding, and with stride = 2. * [lower_convs_to_matmul] removed old assertion [test_conv_lowering] added more dilation values to test cases. Dilation values of {1, 2, 3, 4} are tested. * [im2col] function compute_conv_output_dim can now be called in case non-equal and equal padding is assumed. [test_im2col] changed function call to compute_conv_output_dim [test_conv_lowering] changed function call to compute_conv_output_dim * [im2col] function compute_conv_output_dim can now be called in case non-equal and equal padding is assumed. [test_im2col] changed function call to compute_conv_output_dim. [test_conv_lowering] changed function call to compute_conv_output_dim. [lower_convs_to_matmul] removed old assertion. * [im2col]: changed how a square kernel is instantiated. [lower_convs_to_matmul]: changed how the kernel size attribute is read (based on how a square kernel is instantiated). * [im2col]: changed how a square kernel is instantiated. [lower_convs_to_matmul]: changed how the kernel size attribute is read (based on how a square kernel is instantiated). * [im2col]: changed how a square kernel is instantiated. [lower_convs_to_matmul]: changed how the kernel size attribute is read (based on how a square kernel is instantiated). * [im2col]: minor change in style. * [test_conv_lowering]: minor fix for test case depthwise and regular convolutions * [im2col]: minor style adjustment. [test_conv_lowering]: merged test functions into one test function. * Support for non-square input images and kernels for im2col node (#20) * [im2col.py]: support for non-square input images and kernels, [test_im2col]: added/modified several test cases for (non-)square images and kernels * [im2col]: support for non-square input images and kernels, [test_im2col]: added/modified several test cases for (non-)square images and kernels * [test_general_transformation]: changed kernel_size attribute to list instead of integer as required by im2col node * [base]: changed how the "ints" field of AttributeProto set. [test_general_transformation]: changed the type of the kernel_size attribute to list of integers * removed unused import * [base]: added support for writing repeated fields in AttributeProto * minor style changes * [im2col, test_im2col]: added support for non-equal padding * [lower_convs_to_matmul]: added support for non-square input images and kernels and non-equal padding. [test_conv_lowering]: added/modified test cases for non-equal padding, depthwise convolution and 'standard' convolution. * [test_conv_lowering]: included 1D depthwise and regular convolutions in tests * Revert "[test_conv_lowering]: included 1D depthwise and regular convolutions in tests" This reverts commit 3ff449c42d709e640ca904c41a241bb94fc9e335 * Revert "[lower_convs_to_matmul]: added support for non-square input images and kernels and non-equal padding." This reverts commit 15e34ed8d07d4a55996f162bd3bd1aa24b33c3ac. * Revert "[im2col, test_im2col]: added support for non-equal padding" This reverts commit c524020ee8a7b363eb0c30d70cf21332e9c73678. * [im2col]: changed how a square kernel is instantiated. [lower_convs_to_matmul]: changed how the kernel size attribute is read (based on how a square kernel is instantiated). * [im2col]: minor change in style. * [Im2Col] style fixes and comments Co-authored-by: Yaman Umuroglu * Update AUTHORS.rst * Support for non-square input images and kernels in LowerConvsToMatMul transformation (#16) * [im2col.py]: support for non-square input images and kernels, [test_im2col]: added/modified several test cases for (non-)square images and kernels * [im2col]: support for non-square input images and kernels, [test_im2col]: added/modified several test cases for (non-)square images and kernels * [test_general_transformation]: changed kernel_size attribute to list instead of integer as required by im2col node * [base]: changed how the "ints" field of AttributeProto set. [test_general_transformation]: changed the type of the kernel_size attribute to list of integers * removed unused import * [base]: added support for writing repeated fields in AttributeProto * minor style changes * [im2col, test_im2col]: added support for non-equal padding * [lower_convs_to_matmul]: added support for non-square input images and kernels and non-equal padding. [test_conv_lowering]: added/modified test cases for non-equal padding, depthwise convolution and 'standard' convolution. * [test_conv_lowering]: included 1D depthwise and regular convolutions in tests * Revert "[test_conv_lowering]: included 1D depthwise and regular convolutions in tests" This reverts commit 3ff449c42d709e640ca904c41a241bb94fc9e335 * Revert "[lower_convs_to_matmul]: added support for non-square input images and kernels and non-equal padding." This reverts commit 15e34ed8d07d4a55996f162bd3bd1aa24b33c3ac. * Revert "[im2col, test_im2col]: added support for non-equal padding" This reverts commit c524020ee8a7b363eb0c30d70cf21332e9c73678. * [im2col] function compute_conv_output_dim can now be called in case non-equal and equal padding is assumed. [test_im2col] changed function call to compute_conv_output_dim. [test_conv_lowering] changed function call to compute_conv_output_dim. [lower_convs_to_matmul] removed old assertion. * [im2col]: changed how a square kernel is instantiated. [lower_convs_to_matmul]: changed how the kernel size attribute is read (based on how a square kernel is instantiated). * [im2col]: changed how a square kernel is instantiated. [lower_convs_to_matmul]: changed how the kernel size attribute is read (based on how a square kernel is instantiated). * [im2col]: minor change in style. * [test_conv_lowering]: minor fix for test case depthwise and regular convolutions * Support for non-square input images and kernels for im2col node (#20) * [im2col.py]: support for non-square input images and kernels, [test_im2col]: added/modified several test cases for (non-)square images and kernels * [im2col]: support for non-square input images and kernels, [test_im2col]: added/modified several test cases for (non-)square images and kernels * [test_general_transformation]: changed kernel_size attribute to list instead of integer as required by im2col node * [base]: changed how the "ints" field of AttributeProto set. [test_general_transformation]: changed the type of the kernel_size attribute to list of integers * removed unused import * [base]: added support for writing repeated fields in AttributeProto * minor style changes * [im2col, test_im2col]: added support for non-equal padding * [lower_convs_to_matmul]: added support for non-square input images and kernels and non-equal padding. [test_conv_lowering]: added/modified test cases for non-equal padding, depthwise convolution and 'standard' convolution. * [test_conv_lowering]: included 1D depthwise and regular convolutions in tests * Revert "[test_conv_lowering]: included 1D depthwise and regular convolutions in tests" This reverts commit 3ff449c42d709e640ca904c41a241bb94fc9e335 * Revert "[lower_convs_to_matmul]: added support for non-square input images and kernels and non-equal padding." This reverts commit 15e34ed8d07d4a55996f162bd3bd1aa24b33c3ac. * Revert "[im2col, test_im2col]: added support for non-equal padding" This reverts commit c524020ee8a7b363eb0c30d70cf21332e9c73678. * [im2col]: changed how a square kernel is instantiated. [lower_convs_to_matmul]: changed how the kernel size attribute is read (based on how a square kernel is instantiated). * [im2col]: minor change in style. * [Im2Col] style fixes and comments Co-authored-by: Yaman Umuroglu * Update AUTHORS.rst Co-authored-by: Yaman Umuroglu Co-authored-by: Yaman Umuroglu * Update quantavgpool2d.py (#22) Add "Copyright (c) 2021 Xilinx, Inc" heather * [batchnorm_to_affine]: epsilon value is now read out from the attributes. (#21) [test_batchnorm_to_affine]: added a test case for various epsilon values. * Added 3D to 4D (tensor) transformation (#19) * [im2col.py]: support for non-square input images and kernels, [test_im2col]: added/modified several test cases for (non-)square images and kernels * [im2col]: support for non-square input images and kernels, [test_im2col]: added/modified several test cases for (non-)square images and kernels * [test_general_transformation]: changed kernel_size attribute to list instead of integer as required by im2col node * [base]: changed how the "ints" field of AttributeProto set. [test_general_transformation]: changed the type of the kernel_size attribute to list of integers * removed unused import * [base]: added support for writing repeated fields in AttributeProto * minor style changes * [im2col, test_im2col]: added support for non-equal padding * [lower_convs_to_matmul]: added support for non-square input images and kernels and non-equal padding. [test_conv_lowering]: added/modified test cases for non-equal padding, depthwise convolution and 'standard' convolution. * [test_conv_lowering]: included 1D depthwise and regular convolutions in tests * Revert "[test_conv_lowering]: included 1D depthwise and regular convolutions in tests" This reverts commit 3ff449c42d709e640ca904c41a241bb94fc9e335 * Revert "[lower_convs_to_matmul]: added support for non-square input images and kernels and non-equal padding." This reverts commit 15e34ed8d07d4a55996f162bd3bd1aa24b33c3ac. * Revert "[im2col, test_im2col]: added support for non-equal padding" This reverts commit c524020ee8a7b363eb0c30d70cf21332e9c73678. * [im2col] added support for dilations = 2 for 2D and 1D images. Dilation value must be equal along each axis. [test_im2col] added several test cases in case dilations = 2 (a.o. cases where image and kernel are 2D and 1D, with and without padding, and with stride = 2). [lower_convs_to_matmul] added support for dilation value. Dilation value must be equal along each axis. [test_conv_lowering] added test case for dilations = 2 for 2D and 1D images and kernels, with and without padding, and with stride = 2. * [lower_convs_to_matmul] removed old assertion [test_conv_lowering] added more dilation values to test cases. Dilation values of {1, 2, 3, 4} are tested. * [im2col] function compute_conv_output_dim can now be called in case non-equal and equal padding is assumed. [test_im2col] changed function call to compute_conv_output_dim [test_conv_lowering] changed function call to compute_conv_output_dim * [im2col] function compute_conv_output_dim can now be called in case non-equal and equal padding is assumed. [test_im2col] changed function call to compute_conv_output_dim. [test_conv_lowering] changed function call to compute_conv_output_dim. [lower_convs_to_matmul] removed old assertion. * [im2col]: minor fix with assumption on kernel dimension [lower_convs_to_matmul]: minor fix with assumption on kernel dimension * [change_3d_tensors_to_4d]: added new transformation that transforms 3D tensors to 4D and changes the nodes accordingly [test_4d_conversion]: test function for 3D to 4D tensor transformation * [change_3d_tensors_to_4d]: added new transformation that changes 3D tensors to 4D. [test_4d_conversion]: added a test case for the 3D to 4D transformation. * [change_3d_tensors_to_4d]: added 3D to 4D transformation (for QuartzNet). [test_4d_conversion]: added test case for 3D to 4D transform. * [change_3d_tensors_to_4d]: changed how an invalid graph is handled. [test_4d_conversion]: changed the test case for an invalid graph. * [im2col]: changed how a square kernel is instantiated. [lower_convs_to_matmul]: changed how the kernel size attribute is read (based on how a square kernel is instantiated). * [im2col]: changed how a square kernel is instantiated. [lower_convs_to_matmul]: changed how the kernel size attribute is read (based on how a square kernel is instantiated). * [im2col]: changed how a square kernel is instantiated. [lower_convs_to_matmul]: changed how the kernel size attribute is read (based on how a square kernel is instantiated). * [im2col]: minor change in style. * [im2col]: minor style change and changed the way how a square kernel is instantiated. * [test_conv_lowering]: merged tests for depthwise and standard convolutions. * [test_conv_lowering]: minor fix for test case depthwise and regular convolutions * [im2col]: minor style adjustment. [test_conv_lowering]: merged test functions into one test function. * [change_3d_tensors_to_4d]: style fixes and comments. [test_4d_converions]: rearranged code. * [Transform] check invalid node list length * [change_3d_tensors_to_4d]: rearranged the code to make it more readable. Co-authored-by: Yaman Umuroglu * [Docs] update tutorials * [Util] experimental: fast_mode data packing for binary (#24) * Generic partitioning feature (#23) * Add basic partitioning functionality * Mount build dir within docker container * Support for non-linear models and multi in/out partitions * Remove dataflowpartition custom op from finn-base * Fix temporary build dir for CI * Fix docstring * [create_generic_partitions]: minor modification, removed redundant output value_info entries. (#26) * [extend_partition]: added a new transformation ExtendPartition. (#27) [test_extend_partition]: added a test case for the new transformation. * Added support for non-equal strides along different axes (#25) * [im2col]: added support for non-equal strides along different axes and cleaned up the code. [lower_convs_to_matmul]: added support for non-equal strides along different axes and cleaned up the code. [test_conv_lowering]: added test case for non-equal strides along different axes. * [im2col]: minor fix. [test_im2col]: added test case for non-equal strides along different axes. * Changes for supporting vitis_hls (#28) * [Refactor] split up RTL/HLS-related utils * [Util] rename to CallHLS and allow specifying vivado_hls/vitis_hls * [Util] more flexible stream naming in rtlsim_multi_io * Changes for supporting non-equal dilation (#29) * added support for non-equal dilation value along (H, W) dimension * added test cases for non-equal dilation configurations * appending dilation value along dummy dimension correctly (i.e. with a '1') * changed tensor sparsity annotation for consistency * Support infer_datatype for flatten layer (#30) * Support infer_datatype for flatten layer * [InferDT] add more identity op types for datatype inference * [Lint] fix linting issues Co-authored-by: Yaman Umuroglu * Update AUTHORS.rst * Create python-publish.yml * Add ZCU111 board to part map (#32) * Update AUTHORS.rst Co-authored-by: Mirza Mrahorovic <34712307+mmrahorovic@users.noreply.github.com> Co-authored-by: jalezeta <51440887+jalezeta@users.noreply.github.com> Co-authored-by: Felix Jentzsch <45395194+fpjentzsch@users.noreply.github.com> * Update python-publish.yml * Update python-publish.yml * Lint Co-authored-by: Mirza Mrahorovic <34712307+mmrahorovic@users.noreply.github.com> Co-authored-by: jalezeta <51440887+jalezeta@users.noreply.github.com> Co-authored-by: Felix Jentzsch <45395194+fpjentzsch@users.noreply.github.com> * Inference cost: JSON export, parameter for output onnx * Added support for cost estimation for upsampling Co-authored-by: Yaman Umuroglu Co-authored-by: Felix Jentzsch <45395194+fpjentzsch@users.noreply.github.com> Co-authored-by: Mirza Mrahorovic <34712307+mmrahorovic@users.noreply.github.com> Co-authored-by: jalezeta <51440887+jalezeta@users.noreply.github.com> Co-authored-by: Felix Jentzsch --- .github/workflows/python-publish.yml | 31 +++++++++++++++++++++++ AUTHORS.rst | 6 +++-- src/finn/analysis/inference_cost.py | 37 ++++++++++++++++++++++++++++ src/finn/util/basic.py | 2 ++ src/finn/util/inference_cost.py | 18 ++++++++------ 5 files changed, 85 insertions(+), 9 deletions(-) create mode 100644 .github/workflows/python-publish.yml diff --git a/.github/workflows/python-publish.yml b/.github/workflows/python-publish.yml new file mode 100644 index 0000000..9087804 --- /dev/null +++ b/.github/workflows/python-publish.yml @@ -0,0 +1,31 @@ +# This workflow will upload a Python Package using Twine when a release is created +# For more information see: https://help.github.com/en/actions/language-and-framework-guides/using-python-with-github-actions#publishing-to-package-registries + +name: Upload Python Package + +on: + release: + types: [created] + +jobs: + deploy: + + runs-on: ubuntu-latest + + steps: + - uses: actions/checkout@v2 + - name: Set up Python + uses: actions/setup-python@v2 + with: + python-version: '3.x' + - name: Install dependencies + run: | + python -m pip install --upgrade pip + pip install setuptools wheel twine + - name: Build and publish + env: + TWINE_USERNAME: __token__ + TWINE_PASSWORD: ${{ secrets.PYPI_API_TOKEN }} + run: | + python setup.py sdist + twine upload dist/* diff --git a/AUTHORS.rst b/AUTHORS.rst index 4832a4c..14346eb 100644 --- a/AUTHORS.rst +++ b/AUTHORS.rst @@ -2,10 +2,12 @@ Contributors ============ -* Yaman Umuroglu (maintainer) -* Sambhav Jain (maintainer) +* Yaman Umuroglu (@maltanar) (maintainer) +* Sambhav Jain (@sjain-stanford) * Jakoba Petri-Koenig (@auphelia) * Lucian Petrica (@quetric) * Tobias Alonso (@Tobi-Alonso) * Hendrik Borras (@HenniOVP) * Mirza Mrahorovic (@mmrahorovic) +* Felix Paul Jentzsch (@felixpj) +* Jon Ander Lezeta (@jalezeta) diff --git a/src/finn/analysis/inference_cost.py b/src/finn/analysis/inference_cost.py index 85c1765..1dc6a17 100644 --- a/src/finn/analysis/inference_cost.py +++ b/src/finn/analysis/inference_cost.py @@ -165,6 +165,42 @@ def inference_cost_matmul(model, node, discount_sparsity): return ret +def inference_cost_upsample(model, node, discount_sparsity): + # extract info about the upsampling kernel attributes + mode = get_by_name(node.attribute, "mode").s.decode("utf-8") + scales_tensor = node.input[1] + scales_initializer = model.get_initializer(scales_tensor) + + # extract info from tensor shapes and datatypes + (i_dtype, scale_dtype, o_dtype) = get_node_tensor_dtypes(model, node) + (i_shape, scale_shape, o_shape) = get_node_tensor_shapes(model, node) + bsize = i_shape[0] + ifm_ch = i_shape[1] + ofm_pix_total = np.prod(o_shape[2:]) + + # MAC calculation + if mode == "nearest": + # No calculation involved, since data is just copied over multiple times + n_macs = 0 + elif mode == "linear": + # Data gets linearly interpolated in each dimension + # Two MACs per dimension and output pixel assumed + n_dim_scaling = np.sum(scales_initializer > 1) + n_macs = 2 * n_dim_scaling * ofm_pix_total * ifm_ch * bsize + else: + raise ValueError(f"Upsampling mode {mode} not supported for estimation.") + + # Mem calculation + o_mem = np.prod(o_shape) + idt_name = i_dtype.name + odt_name = o_dtype.name + mac_op_type_str = "op_mac_%s_%s" % (idt_name, idt_name) + o_mem_type_str = "mem_o_%s" % (odt_name) + + ret = {mac_op_type_str: n_macs, o_mem_type_str: o_mem} + return ret + + def inference_cost(model, discount_sparsity=True): "Ensure all nodes have unique names prior to calling this analysis pass." @@ -193,6 +229,7 @@ def inference_cost(model, discount_sparsity=True): "Conv": inference_cost_conv, "MatMul": inference_cost_matmul, "Gemm": inference_cost_matmul, + "Upsample": inference_cost_upsample, } for node in model.graph.node: if node.op_type in inference_cost_fxn_map.keys(): diff --git a/src/finn/util/basic.py b/src/finn/util/basic.py index 3781470..cacb5d4 100644 --- a/src/finn/util/basic.py +++ b/src/finn/util/basic.py @@ -44,6 +44,7 @@ pynq_part_map["Pynq-Z2"] = "xc7z020clg400-1" pynq_part_map["ZCU102"] = "xczu9eg-ffvb1156-2-e" pynq_part_map["ZCU104"] = "xczu7ev-ffvc1156-2-e" +pynq_part_map["ZCU111"] = "xczu28dr-ffvg1517-2-e" # native AXI HP port width (in bits) for PYNQ boards pynq_native_port_width = dict() @@ -52,6 +53,7 @@ pynq_native_port_width["Ultra96"] = 128 pynq_native_port_width["ZCU102"] = 128 pynq_native_port_width["ZCU104"] = 128 +pynq_native_port_width["ZCU111"] = 128 # Alveo device and platform mappings alveo_part_map = dict() diff --git a/src/finn/util/inference_cost.py b/src/finn/util/inference_cost.py index 7c3628b..0ebcec9 100644 --- a/src/finn/util/inference_cost.py +++ b/src/finn/util/inference_cost.py @@ -68,20 +68,20 @@ def compute_mem_bits(inf_cost_dict, filter_string="mem_w"): def inference_cost( model_filename, *, - output_json="inference_cost.json", + output_json=None, + output_onnx=None, preprocess=True, - save_final=True, discount_sparsity=True ): """Print the inference cost estimate metric for given ONNX model. Supports the Quant op for weight/activation quantization. :param model_filename: Filename for ONNX model - :param output_json: Filename for output JSON with detailed inference cost dict + :param output_json: Optional JSON filename to save the inference cost dict + :param output_onnx: Optional ONNX filename to save the final model after any + preprocessing :param preprocess: If set, run preprocessing steps such as shape inference, datatype inference and constant folding. Strongly recommended. - :param save_final: If set, save the final ONNX model after any preprocessing - as final.onnx :param discount_sparsity: If set, will discount op cost of MAC ops with a constant zero weight, and the mem cost of constant zero weights. """ @@ -100,8 +100,8 @@ def inference_cost( model = model.transform(InferDataTypes()) model = model.transform(GiveUniqueNodeNames()) model = model.transform(GiveReadableTensorNames()) - if save_final: - model.save("final.onnx") + if output_onnx is not None: + model.save(output_onnx) ret = model.analysis(lambda x: infca.inference_cost(x, discount_sparsity)) bops = compute_bops(ret) mem_w_bits = compute_mem_bits(ret, "mem_w") @@ -114,6 +114,10 @@ def inference_cost( ret["unsupported"] = str(ret["unsupported"]) print(json.dumps(ret, sort_keys=True, indent=2)) + if output_json is not None: + with open(output_json, "w") as f: + json.dump(ret, f, sort_keys=True, indent=2) + def main(): clize.run(inference_cost)