Add LoRA option for VisionTransformer

[wonjuleee]

Summary

• Add LoRA option for ViT models
• LoRA degrades the performance of ViT-Tiny, but it shows some gains on both iter time and e2e time for larger models. Specifically, while LoRA shows 1, 9% gain of iter time with ViT-base and ViT-Large, respectively, but this reduces the overall training epoch, resulting in 17, 33% decreases in e2e time perspectives.

vit-tiny		test/acc	train/e2e	train/iter	train/epoch
wo LoRA	medium	0.8102	96.1244	0.0670	29.6000
	large	0.8642	339.1453	0.0769	18.0000
w LoRA	medium	0.8324	114.3385	0.0777	33.0000
	large	0.8218	358.7889	0.0783	19.0000
vit-base		test/acc	train/e2e	train/iter	train/epoch
wo LoRA	medium	0.8521	191.8413	0.1885	21.8000
	large	0.8808	667.1776	0.1920	16.2000
w LoRA	medium	0.8995	181.4821	0.1854	23.4000
	large	0.9144	468.3877	0.1909	11.4000
vit-large		test/acc	train/e2e	train/iter	train/epoch
wo LoRA	medium	0.8657	423.8398	0.5561	17.4000
	large	0.9011	1703.7958	0.5623	14.6000
w LoRA	medium	0.9193	308.9256	0.5037	13.4000
	large	0.9291	1022.9644	0.5096	9.2000

Checklist

• I have added unit tests to cover my changes.​
• I have added integration tests to cover my changes.​
• I have ran e2e tests and there is no issues.
• I have added the description of my changes into CHANGELOG in my target branch (e.g., CHANGELOG in develop).​
• I have updated the documentation in my target branch accordingly (e.g., documentation in develop).
• I have linked related issues.

License

• I submit my code changes under the same Apache License that covers the project.
  Feel free to contact the maintainers if that's a concern.
• I have updated the license header for each file (see an example below).

# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0

[sovrasov]

@wonjuleee please merge this branch with develop to enable integration tests CI runner