Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[tracking] E2EShark IREE Tests - Onnx Op - Compile #563

Open
8 of 25 tasks
jinchen62 opened this issue Mar 28, 2024 · 0 comments
Open
8 of 25 tasks

[tracking] E2EShark IREE Tests - Onnx Op - Compile #563

jinchen62 opened this issue Mar 28, 2024 · 0 comments
Assignees

Comments

@jinchen62
Copy link
Contributor

jinchen62 commented Mar 28, 2024

Tracking fixes of E2E IREE tests. Below is the list of ops for which OnnxToTorch and TorchToLinalg lowerings exist but failed to compile on the following tests.

To reproduce issue:
build venv following https://github.com/nod-ai/SHARK-TestSuite/tree/main/iree_tests#common-venv-setup-with-deps
run iree-compile iree_tests/onnx/node/generated/TEST_NAME/model.mlir -o test.vmfb --iree-hal-target-backends=llvm-cpu --mlir-print-ir-after-all

When picking up any failures below, please cross check #215, #347, #450, #566 to see if there is overlap issue.

  • Math ops lowering and runtime issue iree-org/iree#17184
    • test_acos
    • test_acos_example
    • test_acosh
    • test_acosh_example
    • test_asin
    • test_asin_example
    • test_asinh
    • test_asinh_example
    • test_atanh
    • test_atanh_example
    • test_cosh
    • test_cosh_example
    • test_sinh
    • test_sinh_example
  • onnx.ArgMax #635
    • test_argmax_default_axis_example_select_last_index
    • test_argmax_default_axis_random_select_last_index
    • test_argmax_keepdims_example_select_last_index
    • test_argmax_keepdims_random_select_last_index
    • test_argmax_negative_axis_keepdims_example_select_last_index
    • test_argmax_negative_axis_keepdims_random_select_last_index
    • test_argmax_no_keepdims_example_select_last_index
    • test_argmax_no_keepdims_random_select_last_index
  • onnx.ArgMin #648
    • test_argmin_default_axis_example_select_last_index
    • test_argmin_default_axis_random_select_last_index
    • test_argmin_keepdims_example_select_last_index
    • test_argmin_keepdims_random_select_last_index
    • test_argmin_negative_axis_keepdims_example_select_last_index
    • test_argmin_negative_axis_keepdims_random_select_last_index
    • test_argmin_no_keepdims_example_select_last_index
    • test_argmin_no_keepdims_random_select_last_index
  • onnx.AveragePool
    • test_averagepool_2d_dilations
    • test_averagepool_2d_precomputed_same_upper
    • test_averagepool_2d_same_lower
    • test_averagepool_2d_same_upper
    • test_averagepool_3d_dilations_large_count_include_pad_is_0_ceil_mode_is_False
    • test_averagepool_3d_dilations_large_count_include_pad_is_0_ceil_mode_is_True
    • test_averagepool_3d_dilations_large_count_include_pad_is_1_ceil_mode_is_False
    • test_averagepool_3d_dilations_large_count_include_pad_is_1_ceil_mode_is_True
    • test_averagepool_3d_dilations_small
    • test_averagepool_3d_default
  • onnx.BatchNormalization
    • test_batchnorm_epsilon_training_mode
    • test_batchnorm_example_training_mode
  • onnx.Bernoulli
    • test_bernoulli_seed
  • Missing dtype support for OnnxToTorch llvm/torch-mlir#3255
    • test_cast_FLOAT16_to_FLOAT8E4M3FN
    • test_cast_FLOAT16_to_FLOAT8E4M3FNUZ
    • test_cast_FLOAT16_to_FLOAT8E5M2
    • test_cast_FLOAT16_to_FLOAT8E5M2FNUZ
    • test_cast_FLOAT_to_FLOAT8E4M3FN
    • test_cast_FLOAT_to_FLOAT8E4M3FNUZ
    • test_cast_FLOAT_to_FLOAT8E5M2
    • test_cast_FLOAT_to_FLOAT8E5M2FNUZ
    • test_cast_FLOAT_to_STRING
    • test_cast_FLOAT8E4M3FNUZ_to_FLOAT
    • test_cast_FLOAT8E4M3FNUZ_to_FLOAT16
    • test_cast_FLOAT8E4M3FN_to_FLOAT
    • test_cast_FLOAT8E4M3FN_to_FLOAT16
    • test_cast_FLOAT8E5M2FNUZ_to_FLOAT
    • test_cast_FLOAT8E5M2FNUZ_to_FLOAT16
    • test_cast_FLOAT8E5M2_to_FLOAT
    • test_cast_FLOAT8E5M2_to_FLOAT16
    • test_cast_no_saturate_FLOAT16_to_FLOAT8E4M3FN
    • test_cast_no_saturate_FLOAT16_to_FLOAT8E4M3FNUZ
    • test_cast_no_saturate_FLOAT16_to_FLOAT8E5M2
    • test_cast_no_saturate_FLOAT16_to_FLOAT8E5M2FNUZ
    • test_cast_no_saturate_FLOAT_to_FLOAT8E4M3FN
    • test_cast_no_saturate_FLOAT_to_FLOAT8E4M3FNUZ
    • test_cast_no_saturate_FLOAT_to_FLOAT8E5M2
    • test_cast_no_saturate_FLOAT_to_FLOAT8E5M2FNUZ
    • test_castlike_FLOAT_to_FLOAT8E4M3FNUZ_expanded
    • test_castlike_FLOAT_to_FLOAT8E4M3FN_expanded
    • test_castlike_FLOAT_to_FLOAT8E5M2FNUZ_expanded
    • test_castlike_FLOAT_to_FLOAT8E5M2_expanded
    • test_castlike_FLOAT_to_STRING_expanded
    • test_castlike_FLOAT8E4M3FN_to_FLOAT_expanded
    • test_castlike_FLOAT8E4M3FNUZ_to_FLOAT_expanded
    • test_castlike_FLOAT8E5M2_to_FLOAT_expanded
    • test_castlike_FLOAT8E5M2FNUZ_to_FLOAT_expanded
    • test_castlike_FLOAT_to_FLOAT8E4M3FN
    • test_castlike_FLOAT_to_FLOAT8E4M3FNUZ
    • test_castlike_FLOAT_to_FLOAT8E5M2
    • test_castlike_FLOAT_to_FLOAT8E5M2FNUZ
    • test_castlike_FLOAT_to_STRING
    • test_castlike_FLOAT8E4M3FNUZ_to_FLOAT
    • test_castlike_FLOAT8E4M3FN_to_FLOAT
    • test_castlike_FLOAT8E5M2FNUZ_to_FLOAT
    • test_castlike_FLOAT8E5M2_to_FLOAT
  • onnx.Conv
    • test_conv_with_autopad_same
  • onnx.ConvTranspose
    • test_convtranspose_autopad_same
    • test_convtranspose_kernel_shape
    • test_convtranspose_output_shape
  • onnx.CumSum #662
    • test_cumsum_1d_exclusive
    • test_cumsum_1d_reverse
    • test_cumsum_1d_reverse_exclusive
  • onnx.DequantizeLinear
    • test_dequantizelinear_axis
    • test_dequantizelinear_blocked
    • test_dequantizelinear_e4m3fn
    • test_dequantizelinear_e4m3fn_zero_point
    • test_dequantizelinear_e5m2
    • test_dequantizelinear_int16
    • test_dequantizelinear_uint16
  • onnx.Expand
    • test_affine_grid_3d_align_corners_expanded
    • test_affine_grid_3d_expanded
  • onnx.GridSample #649
    • test_gridsample_bicubic
    • test_gridsample_bicubic_align_corners_0_additional_1
    • test_gridsample_bicubic_align_corners_1_additional_1
    • test_gridsample_border_padding
    • test_gridsample_nearest
    • test_gridsample_nearest_align_corners_0_additional_1
    • test_gridsample_nearest_align_corners_1_additional_1
    • test_gridsample_reflection_padding
    • test_gridsample_volumetric_bilinear_align_corners_0
    • test_gridsample_volumetric_bilinear_align_corners_1
    • test_gridsample_volumetric_nearest_align_corners_0
    • test_gridsample_volumetric_nearest_align_corners_1
    • test_gridsample_zeros_padding
  • onnx.Max
    • test_max_one_input
  • onnx.MaxPool
    • test_maxpool_1d_default
    • test_maxpool_2d_precomputed_same_upper
    • test_maxpool_2d_same_lower
    • test_maxpool_2d_same_upper
    • test_maxpool_2d_uint8
    • test_maxpool_with_argmax_2d_precomputed_strides
  • onnx.Min
    • test_min_one_input
  • onnx.Mod
    • test_mod_mixed_sign_float16
    • test_mod_mixed_sign_float32
    • test_mod_mixed_sign_float64
  • onnx.Pad
    • test_center_crop_pad_crop_axes_chw_expanded
    • test_center_crop_pad_crop_axes_hwc_expanded
    • test_center_crop_pad_crop_negative_axes_hwc_expanded
    • test_constant_pad_axes
    • test_constant_pad_negative_axes
    • test_reflect_pad
    • test_wrap_pad
    • test_edge_pad
  • onnx.QuantizeLinear
    • test_quantizelinear_axis
    • test_quantizelinear_blocked
    • test_quantizelinear_e4m3fn
    • test_quantizelinear_e5m2
    • test_quantizelinear_int16
    • test_quantizelinear_uint16
  • onnx.ReduceSum
    • test_nllloss_NCd1_ii_expanded
    • test_nllloss_NCd1_mean_weight_negative_ii_expanded
    • test_nllloss_NCd1_weight_expanded
    • test_nllloss_NCd1_weight_ii_expanded
    • test_nllloss_NCd1d2_no_weight_reduction_mean_ii_expanded
    • test_nllloss_NCd1d2_reduction_sum_expanded
    • test_nllloss_NCd1d2_with_weight_reduction_mean_expanded
    • test_nllloss_NCd1d2_with_weight_reduction_sum_expanded
    • test_nllloss_NCd1d2_with_weight_reduction_sum_ii_expanded
    • test_nllloss_NCd1d2d3_sum_weight_high_ii_expanded
    • test_nllloss_NCd1d2d3d4d5_mean_weight_expanded
  • onnx.ScatterElements
    • test_scatter_elements_with_duplicate_indices
    • test_scatter_elements_with_reduction_max
    • test_scatter_elements_with_reduction_min
  • onnx.Shape
    • test_center_crop_pad_crop_and_pad_expanded
    • test_center_crop_pad_crop_expanded
    • test_center_crop_pad_pad_expanded
  • onnx.Squeeze
    • test_affine_grid_2d_align_corners_expanded
    • test_affine_grid_2d_expanded
  • onnx.Tile #657
    • test_tile
    • test_tile_precomputed
  • onnx.ThresholdedRelu
    • test_thresholdedrelu
    • test_thresholdedrelu_default
    • test_thresholdedrelu_example
@jinchen62 jinchen62 changed the title [tracking] fix OnnxToTorch lowering [tracking] OnnxToTorch op support Mar 28, 2024
@jinchen62 jinchen62 changed the title [tracking] OnnxToTorch op support [tracking] OnnxToTorch Op Support Mar 28, 2024
@jinchen62 jinchen62 changed the title [tracking] OnnxToTorch Op Support [tracking] E2E IREE Tests Compilation Failures Mar 28, 2024
@jinchen62 jinchen62 changed the title [tracking] E2E IREE Tests Compilation Failures [tracking] E2EShark IREE Tests - Onnx Op - Compile Mar 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants