Add "exporting past key values" for TrOCR models

[xenova]

Feature request

Currently, running

optimum-cli export onnx -m microsoft/trocr-base-handwritten out

results in:

Framework not specified. Using pt to export to ONNX.
Some weights of VisionEncoderDecoderModel were not initialized from the model checkpoint at microsoft/trocr-base-handwritten and are newly initialized: ['encoder.pooler.dense.weight', 'encoder.pooler.dense.bias']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
Automatic task detection to image-to-text-with-past.
Could not find image processor class in the image processor config or the model config. Loading based on pattern matching with the model's feature extractor configuration.
/usr/local/lib/python3.10/dist-packages/transformers/models/vit/feature_extraction_vit.py:28: FutureWarning: The class ViTFeatureExtractor is deprecated and will be removed in version 5 of Transformers. Please use ViTImageProcessor instead.
  warnings.warn(
Traceback (most recent call last):
  File "/usr/local/bin/optimum-cli", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.10/dist-packages/optimum/commands/optimum_cli.py", line 163, in main
    service.run()
  File "/usr/local/lib/python3.10/dist-packages/optimum/commands/export/onnx.py", line 232, in run
    main_export(
  File "/usr/local/lib/python3.10/dist-packages/optimum/exporters/onnx/__main__.py", line 399, in main_export
    onnx_config, models_and_onnx_configs = _get_submodels_and_onnx_configs(
  File "/usr/local/lib/python3.10/dist-packages/optimum/exporters/onnx/__main__.py", line 82, in _get_submodels_and_onnx_configs
    onnx_config = onnx_config_constructor(
  File "/usr/local/lib/python3.10/dist-packages/optimum/exporters/onnx/base.py", line 623, in with_past
    return cls(
  File "/usr/local/lib/python3.10/dist-packages/optimum/exporters/onnx/model_configs.py", line 1246, in __init__
    raise ValueError("Exporting past key values is not supported with TrOCR model!")
ValueError: Exporting past key values is not supported with TrOCR model!

Although adding --task image-to-text allows the model to be exported, this should be significantly less efficient that the export with past key values.

cc @fxmarty

Motivation

To run this model with transformers.js

Your contribution

I will integrate TrOCR models into transformers.js when ready.

[luvwinnie]

I'm trying to use this with TrOCR model. Is it possible to use this for custom encoder in the VisionEncoderDecoder?

[baskrahmer]

Related issue, commenting for visibility:
#744

[fxmarty]

closing as duplicate of #744. Will try to add it in the next release

[fxmarty]

Hi, this is fixed in #1456