Update infer and execute API to take prompts from txt file for BS>=1

[quic-mamta]

Update infer and execute API to take prompts from txt file for BS>=1
If user has passed BS=1 and multiple prompts and wants to execute all of them, that is also supported.

1. User can use --prompt or --prompt_txt_file_path, any of these options.
2. With --prompt, multiple prompts to be passed using "|" delimiter
3. Atleast one argument, either --prompt or --prompt_txt_file_path is required.
4. If any one argument is given then it will take inputs from given argument.
5. If both options are given then inputs will be taken from txt file and a warning is given for same.
6. For BS>1, if number of prompts provided in txt file or prompt don't match BS, it throws an error saying mismatch.
7. We currently don't support running in a loop for BS>1

Example commands :

1. for BS=1,
   python -m QEfficient.cloud.infer --model-name gpt2 --batch-size 1 --prompt-len 32 --ctx-len 128 --num-cores 16 --
   device-group [0] --prompt "My name is"

2. for BS>1,
   python -m QEfficient.cloud.infer --model-name gpt2 --batch-size 3 --prompt-len 32 --ctx-len 128 --num-cores 16 --
   device-group [0] --prompt "My name is | The sun rises from | The flat earth theory is the belief that"

3. for BS >=1,
   python -m QEfficient.cloud.infer --model-name gpt2 --batch-size 3 --prompt-len 32 --ctx-len 128 --num-cores 16 --
   device-group [0] --prompts_txt_file_path examples/prompts.txt

[anujgupt-github]

Regarding the description:
"
for bs>1,
python -m QEfficient.cloud.infer --model-name gpt2 --batch-size 3 --prompt-len 32 --ctx-len 128 --num-cores 16 --device-
group '[0]' --inputs_file_path examples/prompts.txt
"

Can a user still use prompts.txt for BS=1, if there's just a single prompt in the txt? If not, that should be allowed too,

[quic-mamta]

Regarding the description: " for bs>1, python -m QEfficient.cloud.infer --model-name gpt2 --batch-size 3 --prompt-len 32 --ctx-len 128 --num-cores 16 --device- group '[0]' --inputs_file_path examples/prompts.txt "

Can a user still use prompts.txt for BS=1, if there's just a single prompt in the txt? If not, that should be allowed too,

Yes, prompts.txt works with BS=1 also, updated the description.

[anujgupt-github]

LGTM

[ochougul]

Can you think of doing a function that takes in input_file_path, qpc_path, prompt as input and decides how to run the qpc on device based on these three parameters.

This should remove duplication of code in two files, and you can call the same function in two files.

[ochougul]

Also, shouldn't we support the same feature via low level API i.e. cloud_ai_100_exec_kv, I think it will be easier if we let this API take in the params qpc_path, device_id, tokenizer which are mandatory parameters to be passed by user,
prompt or input_file_path which will be optional params, but user must pass either one of them.
Then this handles how to execute, this way, nothing changes in the infer, execute files, we just pass an extra parameter in cloud_ai_100_exec_kv and this will handle everything else.

And we support this feature then, via low as well as high level API.

[ochougul]

1. Add XOR condition for exclusively passing either prompt or input_file_path
2. Figure out BS from specializations.json in case of execute
3. Move the logic to exec_kv call, so it doesn't print stats multiple time in case of BS=1, and len(prompt)>1, also clean code.

[quic-mamta]

1. Add XOR condition for exclusively passing either prompt or input_file_path
2. Figure out BS from specializations.json in case of execute
3. Move the logic to exec_kv call, so it doesn't print stats multiple time in case of BS=1, and len(prompt)>1, also clean code.

Updated.

[ochougul]

everything else LGTM.