Android App load local model

[AnswerZhao]

Could you modify the home activity to load the local model instead? The network speed for downloading the model is quite slow. Thank you for sharing this amazing AI project and Android app.

[OmkarThawakar]

Hi @AnswerZhao,

Thank you for interest in our work.

Please refer to llama.cpp (https://github.com/ggerganov/llama.cpp) and llama_cpp_dart (https://github.com/netdur/llama_cpp_dart) for loading local model.

Thanks,

[nonetrix]

You could likely get faster inference using the native android APIs to use NPU but would be a undertaking for sure

[AnswerZhao]

@OmkarThawakar Thank you for the references. I will give it a try.