Explore using the on-device LLM for various tasks

[teolemon]

What

• Explore using on-device LLMs for various tasks. To be clear, this is highly speculative, since only Android has much of the above, and only recent high-end Android have an on-device LLM.

Possible use-cases

Possible use-cases

Beta Give feedback

1. Food intake journaling
2. Ingredient analysis
3. FAQ/Support (chat?)
4. other use-cases TBD
5. Use a multimodal LLM to extract the likely Product name and brand as JSON openfoodfacts-ai#341
   LLMs LLMs - Data extraction product_name +3
   

Caveats

• May drain battery quickly depending on use-cases
• Will need to be limited to recent devices (Pixel 6 and higher, at the current date - January 2024). This might change in the future.

How

• Fully open on-device LLM: https://ai.google/build/project-guides/llm-on-android-with-keras-and-tensorflow-lite
• https://android-developers.googleblog.com/2023/12/leverage-generative-ai-in-your-android-apps.html
• App with LLM template: https://developer.android.com/studio/preview/gemini-template
• Server side LLMs using a SDK: https://ai.google.dev/tutorials/android_quickstart
• On-device Gemini Nano: https://ai.google.dev/tutorials/android_aicore?hl=fr

Originally posted by @teolemon in #4027 (comment)

Part of

• Supercharge the app with artificial intelligence (Tracker) #4027

[teolemon]

@AshAman999 I clarified the situation