optimization profile is missing values for shape input

[OswaldoBornemann]

I tried to convert the onnx model using the following command

shapes = {
    'position' : [(1, 1), (1, 1000), (1, 2000)],
    'inputs_embeds': [(1, 1, 1792), (1, 1000, 1792), (1, 2000, 1792)],
    'past_key_in0': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
    'past_value_in0': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
    'past_key_in1': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
    'past_value_in1': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
    'past_key_in2' : [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
    'past_value_in2': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
    'past_key_in3': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
    'past_value_in3': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
    'past_key_in4': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
    'past_value_in4': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
    'past_key_in5': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
    'past_value_in5': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
    'past_key_in6': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
    'past_value_in6': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
    'past_key_in7': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
    'past_value_in7': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
    'past_key_in8': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
    'past_value_in8': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
    'past_key_in9': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
    'past_value_in9': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
    'past_key_in10': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
    'past_value_in10': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
    'past_key_in11': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
    'past_value_in11': [(1, 16, 1, 112), (1, 16, 1000, 112), (1, 16, 2000, 112)],
    'kv_slice': [(1,), (1,), (1,)],
}

for order, (name, shape_list) in enumerate(shapes.items()):
    print(order, name, shape_list)
    min_shape, opt_shape, max_shape = shape_list
    profile.set_shape(name, min_shape, opt_shape, max_shape)
    

config.add_optimization_profile(profile)
engine = builder.build_serialized_network(network, config)

However, I got the error [TRT] [E] 4: [network.cpp::validate::3584] Error Code 4: Internal Error (kv_slice: optimization profile is missing values for shape input)

[lix19937]

'kv_slice': [(1,), (1,), (1,)], you can uoload the build log of trtexec.

[yuanyao-nv]

If it's a shape tensor you would need to use set_shape_input. See https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/index.html#exe_shape_tensors

[lauvli]

I have encountered the same problem. I have set the input shape, which is also (1,), but I still get an error: Missing Values for Shape Input Tensor.