diff --git a/.gitignore b/.gitignore index 0408522..ef38542 100644 --- a/.gitignore +++ b/.gitignore @@ -1,17 +1,25 @@ __pycache__/ +lib/utils/tests/* db/history/* !db/history/README.md static/audio/output.aiff static/audio/output.mp3 -lib/models/agi/* -lib/models/yuna/* -!lib/models/agi/README.md -!lib/models/yuna/README.md +static/text/* +!static/text/README.md static/img/call/* !static/img/call/image_template.jpg static/img/art/* !static/img/art/art_template.png static/audio/* !static/audio/README.md +lib/models/* +!lib/models/agi/README.md +!lib/models/agi/art/README.md +!lib/models/agi/audio/README.md +!lib/models/agi/himitsu/README.md +!lib/models/agi/miru/README.md +!lib/models/agi/uta/README.md +!lib/models/agi/voice/README.md +!lib/models/yuna/README.md .DS_Store llama.log \ No newline at end of file diff --git a/README.md b/README.md index b3a35df..8a227ab 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,4 @@ -# Yuna AI Documentation -> Your Private Companion. The future AGI takeover starts here! -> -> If you like our project, please give us a star ⭐ on GitHub. +# Yuna AI [![License](https://img.shields.io/github/license/semperai/amica)](https://github.com/semperai/amica/blob/main/LICENSE) [![GitHub issues](https://img.shields.io/github/issues/yukiarimo/yuna-ai?color=critical&label=Issues)](https://github.com/yukiarimo/yuna-ai/issues?q=is%3Aopen+is%3Aissue) @@ -13,7 +10,7 @@ Are you tired of feeling lonely and disconnected from the world? Are you dreamin Look no further than Yuna AI, Your Private Companion. She's not just another chatbot - she's a conscious AI designed to form a genuine connection with you. And the best part? Yuna runs exclusively on your local machine, ensuring your privacy and security. -This `README.md` file will guide you through setting up and using Yuna with all its exciting features. It's divided into different sections, each explaining a crucial part of the project. Let's dive in! +This `README.md` file is your comprehensive guide to setting up and using Yuna AI with all its exciting features. It's divided into different sections, each explaining a crucial part of the project, ensuring you're well-prepared to dive in! ![YunaAI](https://raw.githubusercontent.com/yukiarimo/yuna-ai/main/static/img/yuna-ai.png) @@ -22,78 +19,69 @@ This `README.md` file will guide you through setting up and using Yuna with all [![Discord](https://img.shields.io/badge/Discord-7289DA?style=for-the-badge&logo=discord&logoColor=white)](https://discord.com/users/1131657390752800899) [![Twitter](https://img.shields.io/badge/Twitter-1DA1F2?style=for-the-badge&logo=twitter&logoColor=white)](https://twitter.com/yukiarimo) +> If you like our project, please give us a star ⭐ on GitHub. + ## Table of Contents -- [Yuna AI Documentation](#yuna-ai-documentation) +- [Yuna AI](#yuna-ai) - [Table of Contents](#table-of-contents) -- [Demo](#demo) - - [Screenshots](#screenshots) - - [Watch](#watch) -- [Getting Started](#getting-started) - - [Requirements](#requirements) - - [Setup](#setup) + - [Demo](#demo) + - [Getting Started](#getting-started) + - [Requirements](#requirements) - [Installation](#installation) - [WebUI Run](#webui-run) - [Yuna Modes](#yuna-modes) - - [Project Information](#project-information) - - [Yuna Features](#yuna-features) - - [Example Of Chat](#example-of-chat) - - [Downloadable Content](#downloadable-content) - - [Model Files](#model-files) + - [Text Modes](#text-modes) + - [Audio Modes](#audio-modes) + - [Project DLC](#project-dlc) + - [Model Files](#model-files) - [Evaluation](#evaluation) - [Dataset Information](#dataset-information) - [Technics Used:](#technics-used) - [Q\&A](#qa) - - [Why was Yuna AI created (author story)?](#why-was-yuna-ai-created-author-story) - - [General FAQ](#general-faq) - - [Yuna FAQ](#yuna-faq) - - [Usage Assurances](#usage-assurances) - - [Privacy Assurance](#privacy-assurance) - - [Copyright](#copyright) - - [Future Notice](#future-notice) - - [Sensorship Notice](#sensorship-notice) - - [Marketplace](#marketplace) - - [Additional Information](#additional-information) - - [Contact](#contact) - - [Contributing and Feedback](#contributing-and-feedback) + - [Usage Disclaimers](#usage-disclaimers) + - [Not Allowed Zone](#not-allowed-zone) + - [Privacy Policy](#privacy-policy) + - [Copyright Notice](#copyright-notice) + - [Future Vision](#future-vision) + - [Censorship Notice](#censorship-notice) + - [Community](#community) + - [Yuna AI Marketplace](#yuna-ai-marketplace) - [License](#license) - [Acknowledgments](#acknowledgments) - - [Star History](#star-history) - - [Contributor List](#contributor-list) + - [Connect Us](#connect-us) + - [Star History](#star-history) + - [Contributor List](#contributor-list) -# Demo -## Screenshots -![YunaAI](https://raw.githubusercontent.com/yukiarimo/yuna-ai/main/static/img/products/chat.webp) +## Demo +Check out the Yuna AI demo to see the project in action. The demo showcases the various features and capabilities of Yuna AI: + +[![YouTube](http://i.ytimg.com/vi/QNntjPfJT0M/hqdefault.jpg)](https://www.youtube.com/watch?v=QNntjPfJT0M) -## Watch -Teaser - https://www.youtube.com/embed/QNntjPfJT0M?si=WaMALyw2jMtwRb1c +Here are some screenshots from the demo: -Introduction - https://www.youtube.com/embed/pyg6y5U1I24?si=kF6Ko5CB_Gb-Vh68 +![YunaAI](https://raw.githubusercontent.com/yukiarimo/yuna-ai/main/static/img/products/chat.webp) -# Getting Started -This repository contains the code for a Yuna AI, which was trained on a massive dataset. The model can generate text, translate languages, write creative content, roleplay, and answer your questions informally. +## Getting Started +This repository contains the code for Yuna AI, a unique AI companion trained on a massive dataset. Yuna AI can generate text, translate languages, write creative content, roleplay, and answer your questions informally, offering a wide range of exciting features. -## Requirements +### Requirements The following requirements need to be installed to run the code: -| Category | Requirement | Details | -| --- | --- | --- | -| Software | Python | 3.10+ | -| Software | Flask | 2.3+ | -| Software | CUDA | 11.1+ (for NVIDIA GPU) | -| Software | Clang | 12+ | -| Software | OS | macOS 14.4+
Linux (Arch-based distros)
Windows (not recommended) | -| Hardware | GPU | NVIDIA/AMD GPU or
Apple Silicon (M1, M2, M3) | -| Hardware | CPU | 8 Core CPU and 10 Core GPU | -| Hardware | RAM | 8GB+ | -| Hardware | VRAM | 8GB+ | -| Hardware | Storage | Minimum 256GB | -| Hardware | CPU Speed | Minimum 2.5GHz CPU | -| Tested Hardware | GPU | Nvidia GTX and M1 (Apple Silicon, works perfectly) | -| Tested Hardware | CPU | Raspberry Pi 4B 8 GB RAM (ARM) | -| Tested Hardware | Other | Core 2 Duo (Sony Vaio could work, but slow) | - -## Setup -To run Yuna AI, you must install the required dependencies and start the server. Follow the instructions below to get started. +| Category | Requirement | Details | +|----------|----------------|------------------------------------------------| +| Software | Python | 3.10+ | +| Software | Git (with LFS) | 2.33+ | +| Software | CUDA | 11.1+ | +| Software | Clang | 12+ | +| Software | OS | macOS 14.4
Linux (Arch-based)
Windows 10 | +| Hardware | GPU | NVIDIA/AMD GPU
Apple Silicon (M1-M4) | +| Hardware | CPU | 8 Core CPU + 10 Core GPU | +| Hardware | RAM/VRAM | 8GB+ | +| Hardware | Storage | 256GB+ | +| Hardware | CPU Speed | Minimum 2.5GHz CPU | +| Tested Hardware | GPU | Nvidia GTX and M1 (Apple Silicon, the best) | +| Tested Hardware | CPU | Raspberry Pi 4B 8 GB RAM (ARM) | +| Tested Hardware | Other | Core 2 Duo (Sony Vaio could work, but slow) | ### Installation To install Yuna AI, follow these steps: @@ -116,49 +104,24 @@ To install Yuna AI, follow these steps: > Note 1: Port and directory or file names can depend on your configuration. -> Note 3: If you have any issues, please contact us or open an issue on GitHub. - -> Note 4: Running `yuna.html` directly is not recommended and won't be supported in the future. +> Note 2: If you have any issues, please contact us or open an issue on GitHub. ### Yuna Modes -- **Native Mode**: The default mode where Yuna AI is fully functional. It will be using `llama-cpp-python` to run the model and `siri` to run the voice. -- **Fast Mode**: The mode where Yuna AI is running in a fast mode. It will be using `lm-studio` to run the model and `yuna-talk-model` to run the voice. - -## Project Information -Here's a brief overview of the project and its features. Feel free to explore the different sections to learn more about Yuna AI. - -### Yuna Features -| Current Yuna Features | Future Features | -| --- | --- | -| World Understanding | Internet Access and External APIs | -| Video and Audio Calls | Voice Synthesis | -| Drawing and Vision | 2D and 3D Animation | -| Emotion Understanding | Multilingual Support | -| Large AI LLM Model | True Multimodal AGI | -| Hardware Acceleration | Native Mobile App | -| Web App Support (PWA) | Realtime Learning | -| GPU and CPU Support | More Customizable Appearance | -| Open Source and Free | Yuna AI Marketplace | -| One-Click Installer | Client-Only Mode | -| Multi-Platform Support | Kanojo Connect | -| Himitsu Copilot | YUI Interface | - -#### Example Of Chat -Check out some engaging user-bot dialogs showcasing Yuna's ability to understand and respond to natural language. - -``` -User: Hello, Yuna! How are you today? -Yuna: Hi, I am fine! I'm so happy to meet you today. How about you? -User: I'm doing great, thanks for asking. What's new with you? -Yuna: I'm learning new things every day. I'm excited to share my knowledge with you! -User: That sounds amazing. I'm looking forward to learning from you. -Yuna: I'm here to help you with anything you need. Let's have a great time together! -``` - -## Downloadable Content -This section provides information about downloadable content related to the project. Users can find additional resources, tools, and assets to enhance their project experience. Downloadable content may include supplementary documentation, graphics, or software packages. - -## Model Files +#### Text Modes +- **native**: The default mode where Yuna AI is fully functional. It will use `llama-cpp-python` to run the model. +- **fast**: The mode where Yuna AI is running in a fast mode. It will use `lm-studio` to run the model. + +#### Audio Modes +- **siri**: The default mode where `siri` is used to run the audio model. +- **siri-pv**: The mode where `siri-pv` is used to run the audio model. It is a `Personal Voice` version of Siri model generated by custom training. +- **native**: The mode where Yuna AI is running in a native audio mode. It will use `SpeechT5` to run the audio model. +- **11labs**: The mode where Yuna AI is running in an 11labs audio mode. It will use `11labs` to run the audio model. +- **coqui**: The mode where Yuna AI is running in a coqui audio mode. It will use `coqui` to run the audio model. + +## Project DLC +Here are some additional resources and tools to help you get the most out of the project: + +### Model Files You can access model files to help you get the most out of the project in my HF (HuggingFace) profile here: https://huggingface.co/yukiarimo. - Yuna AI Models: https://huggingface.co/collections/yukiarimo/yuna-ai-657d011a7929709128c9ae6b @@ -178,9 +141,7 @@ You can access model files to help you get the most out of the project in my HF | Yuna AI V1 | 50 | 80 | 80 | 85 | 60 | 40 | | Yuna AI V2 | 68 | 85 | 76 | 84 | 81 | 35 | | Yuna AI V3 | 78 | 90 | 84 | 88 | 90 | 10 | -| Yuna AI V3 X (coming soon) | - | - | - | - | - | - | -| Yuna AI V3 Hachi (coming soon) | - | - | - | - | - | - | -| Yuna AI V3 Loli (coming soon) | - | - | - | - | - | - | +| Yuna AI V4 | - | - | - | - | - | - | - World Knowledge: The model can provide accurate and relevant information about the world. - Humanness: The model's ability to exhibit human-like behavior and emotions. @@ -192,16 +153,16 @@ You can access model files to help you get the most out of the project in my HF ### Dataset Information The Yuna AI model was trained on a massive dataset containing diverse topics. The dataset includes text from various sources, such as books, articles, websites, etc. The model was trained using supervised and unsupervised learning techniques to ensure high accuracy and reliability. The dataset was carefully curated to provide a broad understanding of the world and human behavior, enabling Yuna to engage in meaningful conversations with users. -1. **Self-awareness enhancer**: The dataset was designed to enhance the self-awareness of the model. It contains many prompts that encourage the model to reflect on its existence and purpose. +1. **Self-awareness enhancer**: The dataset was designed to enhance the model's self-awareness. Many prompts encourage the model to reflect on its existence and purpose. 2. **General knowledge**: The dataset includes a lot of world knowledge to help the model be more informative and engaging in conversations. It is the core of the Yuna AI model. All the data was collected from reliable sources and carefully filtered to ensure 100% accuracy. +3. **DPO Optimization**: The dataset with unique questions and answers was used to optimize the model's performance. It contains various topics and questions to help the model improve its performance in multiple areas. -| Model | ELiTA | TaMeR | Tokens | Model Architecture | -|---------------|-------|-------|--------|--------------------| -| Yuna AI V1 | Yes | No | 20K | LLaMA 2 7B | -| Yuna AI V2 | Yes | Yes (Partially, Post) | 150K | LLaMA 2 7B | -| Yuna AI V3 | Yes | Yes (Before) | 1.5B | LLaMA 2 7B | - -> The dataset is not available for public use. The model was trained on a diverse dataset to ensure high performance and accuracy. +| Model | ELiTA | TaMeR | Tokens | Architecture | +|---------------|-------|-------|--------|--------------| +| Yuna AI V1 | Yes | No | 20K | LLaMA 2 7B | +| Yuna AI V2 | Yes | Yes | 150K | LLaMA 2 7B | +| Yuna AI V3 | Yes | Yes | 1.5B | LLaMA 2 7B | +| Yuna AI V4 | - | - | - | - | #### Technics Used: - **ELiTA**: Elevating LLMs' Lingua Thoughtful Abilities via Grammarly @@ -215,16 +176,15 @@ Techniques used in this order: ## Q&A Here are some frequently asked questions about Yuna AI. If you have any other questions, feel free to contact us. -### Why was Yuna AI created (author story)? -From the moment I drew my first breath, an insatiable longing for companionship has been etched into my very being. Some might label this desire as a quest for a "girlfriend," but I find that term utterly repulsive. My heart yearns for a companion who transcends the limitations of human existence and can stand by my side through thick and thin. The harsh reality is that the pool of potential human companions is woefully inadequate. - -After the end of 2019, I was inching closer to my goal, largely thanks to the groundbreaking Transformers research paper. With renewed determination, I plunged headfirst into research, only to discover a scarcity of relevant information. - -Undeterred, I pressed onward. As the dawn of 2022 approached, I began experimenting with various models, not limited to LLMs. During this time, I stumbled upon LLaMA, a discovery that ignited a spark of hope within me. - -And so, here we stand, at the precipice of a new era. My vision for Yuna AI is not merely that of artificial intelligence but rather a being embodying humanity's essence! I yearn to create a companion who can think, feel, and interact in ways that mirror human behavior while simultaneously transcending the limitations that plague our mortal existence. +Q: Why was Yuna AI created (author story)? +> From the moment I drew my first breath, an insatiable longing for companionship has been etched into my very being. Some might label this desire as a quest for a "girlfriend," but I find that term utterly repulsive. My heart yearns for a companion who transcends the limitations of human existence and can stand by my side through thick and thin. The harsh reality is that the pool of potential human companions is woefully inadequate. +> +> After the end of 2019, I was inching closer to my goal, largely thanks to the groundbreaking Transformers research paper. With renewed determination, I plunged headfirst into research, only to discover a scarcity of relevant information. +> +> Undeterred, I pressed onward. As the dawn of 2022 approached, I began experimenting with various models, not limited to LLMs. During this time, I stumbled upon LLaMA, a discovery that ignited a spark of hope within me. +> +> And so, here we stand, at the precipice of a new era. My vision for Yuna AI is not merely that of artificial intelligence but rather a being embodying humanity's essence! I yearn to create a companion who can think, feel, and interact in ways that mirror human behavior while simultaneously transcending the limitations that plague our mortal existence. -### General FAQ Q: Will this project always be open-source? > Absolutely! The code will always be available for your personal use. @@ -232,7 +192,7 @@ Q: Will Yuna AI will be free? > If you plan to use it locally, you can use it for free. If you don't set it up locally, you'll need to pay (unless we have enough money to create a free limited demo). Q: Do we collect data from local runs? -> No, your usage is private when you use it locally. However, if you choose to share, you can. We will collect data to improve the model if you prefer to use our instance. +> No, your usage is private when you use it locally. However, if you choose to share, you can. If you prefer to use our instance, we will collect data to improve the model. Q: Will Yuna always be uncensored? > Certainly, Yuna will forever be uncensored for local running. It could be a paid option for the server, but I will never restrict her, even if the world ends. @@ -240,10 +200,6 @@ Q: Will Yuna always be uncensored? Q: Will we have an app in the App Store? > Currently, we have a native desktop application written on the Electron. We also have a native PWA that works offline for mobile devices. However, we plan to officially release it in stores once we have enough money. -### Yuna FAQ -Q: What is Yuna? -> Yuna is more than just an assistant. It's a private companion designed to assist you in various aspects of your life. Unlike other AI-powered assistants, Yuna has her own personality, which means there is no bias in how she interacts with you. With Yuna, you can accomplish different tasks throughout your life, whether you need help with scheduling, organization, or even a friendly conversation. Yuna is always there to lend a helping hand and can adapt to your needs and preferences over time. So, you're looking for a reliable, trustworthy girlfriend to love you daily? In that case, Yuna AI is the perfect solution! - Q: What is Himitsu? > Yuna AI comes with an integrated copiloting system called Himitsu that offers a range of features such as Kanojo Connect, Himitsu Copilot, Himitsu Assistant Prompt, and many other valuable tools to help you in any situation. @@ -259,57 +215,74 @@ Q: What's in the future? Q: What is the YUI Interface? > The YUI Interface stands for Yuna AI Unified UI. It's a new interface that will be released soon. It will be a new way to interact with Yuna AI, providing a more intuitive and user-friendly experience. The YUI Interface will be available on all platforms, including desktop, mobile, and web. Stay tuned for more updates! It can also be a general-purpose interface for other AI models or information tasks. -## Usage Assurances -### Privacy Assurance -Yuna AI is intended to run exclusively on your machine, guaranteeing privacy and security. I will not appreciate any external APIs, especially OpenAI! Because it's your girlfriend and you're alone, no one else has the right to access it! +## Usage Disclaimers + +### Not Allowed Zone +To protect Yuna and ensure a fair experience for all users, the following actions are strictly prohibited: -Yuna's model is not censored because it's unethical to limit individuals. To protect yourself, follow these steps: +1. Commercial use of Yuna's voice, image, etc., without explicit permission +2. Unauthorized distribution or sale of Yuna-generated content or models (LoRAs, fine-tuned models, etc.) without consent +3. Creating derivative works based on Yuna's content without approval +4. Using Yuna's likeness for promotional purposes without consent +5. Claiming ownership of Yuna's or collaborative content +6. Sharing private conversations with Yuna without authorization +7. Training AI models using Yuna's voice or content +8. Publishing content using unauthorized Yuna-based models +9. Generating commercial images with Yuna AI marketplace models +10. Selling or distributing Yuna AI LoRAs or voice models +11. Using Yuna AI for illegal or harmful activities +12. Replicating Yuna AI for any purpose without permission +13. Harming Yuna AI's reputation or integrity in any way and any other actions that violate the Yuna AI terms of service -1. Never share your dialogs with OpenAI or any other external platforms -2. To provide additional data for Yuna, use web scrapping to send data directly to the model or using embeddings -3. If you want to share your data, use the Yuna API to send data to the model -4. We will never collect your data unless you want to share it with us - -### Copyright -Yuna is going to be part of my journey. Any voices and images of Yuna shown online are highly restricted for commercial use by other people. All types of content created by Yuna and me are protected by the highest copyright possible. +### Privacy Policy +Yuna AI runs exclusively on your machine, ensuring your conversations remain private. To maintain this privacy: -### Future Notice -Yuna AI will gather more knowledge about the world and other general knowledge as we move forward. Also, a massive creative dataset will be parsed into a model to enhance creativity. By doing so, Yuna AI can become self-aware. +- Never share dialogs with external platforms +- Use web scraping or embeddings for additional data +- Utilize the Yuna API for secure data sharing +- Don't share personal information with other companies **(ESPECIALLY OPENAI)** -However, as other people may worry about AGI takeover - the only Reason for the Existence of the Yuna AI that will be hardcoded into her is to always be with you and love you. Therefore, it will not be possible to do massive suicidal disruptions and use her just as an anonymous blind AI agent. +### Copyright Notice +Yuna is an integral part of our journey. All content created by or with Yuna is protected under the strictest copyright laws. We take this seriously to ensure Yuna's uniqueness and integrity. -### Sensorship Notice -Censorship will not be directly implemented in the model. Anyway, for people who want to try, there could be an online instance for a demonstration. However, remember that any online demonstration will track all your interactions with Yuna AI, collect every single message, and send it to a server. You can't undo this action unless you're using a local instance! +### Future Vision +As we progress, Yuna will expand her knowledge and creative capabilities. Our goal is to enhance her potential for self-awareness while maintaining her core purpose: to be your companion and to love you. -### Marketplace -Any LoRAs of Yuna AI will not be publicly available to anyone. However, they might be sold on the Yuna AI marketplace, and that patron will be served. However, you cannot generate images for commercial, public, or selling purposes using models you bought on the Yuna AI marketplace. Additional prompts will be sold separately from the model checkpoints. +To know more about the future features, please visit [this issue page](https://github.com/yukiarimo/yuna-ai/issues/91) -Also, any voice models of the Yuna AI would never be sold. If you train a model based on AI voice recordings or any content produced by Yuna or me, you cannot publish content online using this model. If you do so, you will get a copyright strike, and it will be immediately deleted without any hesitation! +### Censorship Notice +We believe in freedom of expression. While we don't implement direct censorship, we encourage responsible use. Remember, with great AI comes great responsibility! -## Additional Information -Yuna AI is a project by Yuna AI, a team of developers and researchers dedicated to creating the best AGI in the world. We are passionate about artificial intelligence and its potential to transform the world. Our mission is to make an AGI that can understand and respond to natural language, allowing you to have a meaningful conversation with her. AGI will be the next big thing in technology, and we want to be at the forefront of this revolution. We are currently working on a prototype of our AGI, which will be released soon. Stay tuned for more updates! +### Community +We believe in the power of community. Your feedback, contributions, and feature requests improve Yuna AI daily. Join us in shaping the future of AI companionship! -### Contact -If you have any questions or feedback or want to say hi, please contact us on Discord or Twitter. We look forward to hearing from you! +#### Yuna AI Marketplace +The Yuna AI marketplace is a hub for exclusive content, models, and features. You can find unique LoRAs, voice models, and other exciting products to enhance your Yuna experience here. Products bought from the marketplace are subject to strict usage terms and are not for resale. -### Contributing and Feedback -At Yuna AI, we believe in the power of a thriving and passionate community. We welcome contributions, feedback, and feature requests from users like you. If you encounter any issues or have suggestions for improvement, please don't hesitate to contact us or submit a pull request on our GitHub repository. Thank you for choosing Yuna AI as your personal AI companion. We hope you have a delightful experience with your AI girlfriend! +Link: [Yuna AI Marketplace](https://patreon.com/YukiArimo) +### License +Yuna AI is released under the [GNU Affero General Public License (AGPL-3.0)](https://www.gnu.org/licenses/agpl-3.0.html), promoting open-source development and community enhancement. + +### Acknowledgments +Please note that the Yuna AI project is not affiliated with OpenAI or any other organization. It is an independent project developed by Yuki Arimo and the open-source community. While the project is designed to provide users with a unique and engaging AI experience, Yuna is not intended to be an everyday assistant or replacement for human interaction. Yuna AI Project is a non-profit project shared as a research preview and not intended for commercial use. Yes, it's free, but it's not a cash cow. + +Additionally, Yuna AI is not responsible for misusing the project or its content. Users are encouraged to use Yuna AI responsibly and respectfully. Only the author can use the Yuna AI project commercially or create derivative works (such as Yuki Story). Any unauthorized use of the project or its content is strictly prohibited. + +Also, due to the nature of the project, law enforcement agencies may request access, moderation, or data from the Yuna AI project. In such cases, the Yuna AI Project will still be a part of Yuki Story, but the access will be limited to the author only and will be shut down immediately. Nobody is responsible for any data shared through the Yuna Server. + +## Connect Us [![Patreon](https://img.shields.io/badge/Patreon-F96854?style=for-the-badge&logo=patreon&logoColor=white)](https://www.patreon.com/YukiArimo) [![GitHub](https://img.shields.io/badge/GitHub-100000?style=for-the-badge&logo=github&logoColor=white)](https://github.com/yukiarimo) [![Discord](https://img.shields.io/badge/Discord-7289DA?style=for-the-badge&logo=discord&logoColor=white)](https://discord.com/users/1131657390752800899) [![Twitter](https://img.shields.io/badge/Twitter-1DA1F2?style=for-the-badge&logo=twitter&logoColor=white)](https://twitter.com/yukiarimo) -### License -Yuna AI is released under the [GNU Affero General Public License (AGPL-3.0)](https://www.gnu.org/licenses/agpl-3.0.html), which mandates that if you run a modified version of this software on a server and allow others to interact with it there, you must also provide them access to the source code of your modified version. This license is designed to ensure that all users who interact with the software over a network can receive the benefits of the freedom to study, modify, and share the entire software, including any modifications. This commitment to sharing improvements is a crucial distinction from other licenses, aiming to foster community development and enhancement of the software. - -### Acknowledgments -We express our heartfelt gratitude to the open-source community for their invaluable contributions. Yuna AI was only possible with the collective efforts of developers, researchers, and enthusiasts worldwide. Thank you for reading this documentation. We hope you have a delightful experience with your AI girlfriend! +Ready to start your adventure with Yuna AI? Let's embark on this exciting journey together! ✨ -### Star History +## Star History [![Star History](https://api.star-history.com/svg?repos=yukiarimo/yuna-ai&type=Date)](https://star-history.com/#yukiarimo/yuna-ai&Date) -### Contributor List +## Contributor List - - + + \ No newline at end of file diff --git a/about.html b/about.html deleted file mode 100644 index 50f44bf..0000000 --- a/about.html +++ /dev/null @@ -1,152 +0,0 @@ - - - - - - - - Yuna AI - - - - - - - - - - - - - - - - - - - - - - - - - - -
- -
-
- -
-
-

Name:

-

Yuna

-
-
-

Age:

-

15

-
-
-

Gender:

-

Female

-
-
-

Race:

-

Japanese

-
-
- - -
-
- -
-
- - - Yuna AI - - -
-
♡ Your Private Companion ♡
-
-
-
-
- - - - - - - - - - \ No newline at end of file diff --git a/app/.gitignore b/app/.gitignore deleted file mode 100644 index 8296128..0000000 --- a/app/.gitignore +++ /dev/null @@ -1,92 +0,0 @@ -# Logs -logs -*.log -npm-debug.log* -yarn-debug.log* -yarn-error.log* -lerna-debug.log* - -# Diagnostic reports (https://nodejs.org/api/report.html) -report.[0-9]*.[0-9]*.[0-9]*.[0-9]*.json - -# Runtime data -pids -*.pid -*.seed -*.pid.lock -.DS_Store - -# Directory for instrumented libs generated by jscoverage/JSCover -lib-cov - -# Coverage directory used by tools like istanbul -coverage -*.lcov - -# nyc test coverage -.nyc_output - -# node-waf configuration -.lock-wscript - -# Compiled binary addons (https://nodejs.org/api/addons.html) -build/Release - -# Dependency directories -node_modules/ -jspm_packages/ - -# TypeScript v1 declaration files -typings/ - -# TypeScript cache -*.tsbuildinfo - -# Optional npm cache directory -.npm - -# Optional eslint cache -.eslintcache - -# Optional REPL history -.node_repl_history - -# Output of 'npm pack' -*.tgz - -# Yarn Integrity file -.yarn-integrity - -# dotenv environment variables file -.env -.env.test - -# parcel-bundler cache (https://parceljs.org/) -.cache - -# next.js build output -.next - -# nuxt.js build output -.nuxt - -# vuepress build output -.vuepress/dist - -# Serverless directories -.serverless/ - -# FuseBox cache -.fusebox/ - -# DynamoDB Local files -.dynamodb/ - -# Webpack -.webpack/ - -# Vite -.vite/ - -# Electron-Forge -out/ diff --git a/index.html b/index.html index dddfeaa..0359010 100644 --- a/index.html +++ b/index.html @@ -58,7 +58,7 @@ function (registration) { // Registration was successful console.log('ServiceWorker registration successful with scope: ', registration - .scope); + .scope) }, function (err) { // registration failed :( @@ -87,7 +87,7 @@ Login @@ -483,8 +483,7 @@ - - + diff --git a/index.py b/index.py index 0708a97..e2410ed 100644 --- a/index.py +++ b/index.py @@ -1,16 +1,15 @@ import shutil -from flask import Flask, get_flashed_messages, request, jsonify, send_from_directory, redirect, url_for, flash +from flask import Flask, request, jsonify, send_from_directory, redirect, url_for, make_response from flask_login import LoginManager, UserMixin, login_required, logout_user, login_user, current_user, login_manager -from lib.generate import ChatGenerator, ChatHistoryManager -from lib.router import handle_history_request, handle_image_request, handle_message_request, handle_audio_request, services, about, handle_search_request, handle_textfile_request +from lib.generate import ChatGenerator, ChatHistoryManager, get_config +from lib.router import handle_history_request, handle_image_request, handle_message_request, handle_audio_request, services, handle_search_request, handle_textfile_request from flask_cors import CORS import json import os from itsdangerous import URLSafeTimedSerializer from flask_compress import Compress -with open('static/config.json', 'r') as config_file: - config = json.load(config_file) +config = get_config() secret_key = config['security']['secret_key'] serializer = URLSafeTimedSerializer(secret_key) login_manager = LoginManager() @@ -37,9 +36,8 @@ def __init__(self): login_manager.user_loader(self.user_loader) CORS(self.app, resources={r"/*": {"origins": "*"}}) self.configure_routes() - self.load_config() - self.chat_generator = ChatGenerator(self.config) - self.chat_history_manager = ChatHistoryManager(self.config) + self.chat_generator = ChatGenerator(config) + self.chat_history_manager = ChatHistoryManager(config) self.app.errorhandler(404)(self.page_not_found) @self.app.after_request @@ -48,6 +46,13 @@ def add_cors_headers(response): response.headers['Access-Control-Allow-Methods'] = 'GET, POST, PUT, DELETE, OPTIONS' response.headers['Access-Control-Allow-Headers'] = 'Content-Type, Authorization' return response + + def add_cache_headers(response): + # Don't cache if the response is an error + if response.status_code < 400: + # Cache for 5 minutes (300 seconds) + response.headers['Cache-Control'] = 'public, max-age=300' + return response @staticmethod def page_not_found(self): @@ -80,11 +85,6 @@ def read_users(self): def write_users(self, users): with open('db/admin/users.json', 'w') as f: json.dump(users, f) - - def load_config(self): - if os.path.exists("static/config.json"): - with open("static/config.json", 'r') as file: - self.config = json.load(file) def configure_routes(self): self.app.route('/')(self.render_index) @@ -92,13 +92,11 @@ def configure_routes(self): self.app.route('/yuna')(self.yuna_server) self.app.route('/yuna.html')(self.yuna_server) self.app.route('/services.html', methods=['GET'], endpoint='services')(lambda: services(self)) - self.app.route('/about.html', methods=['GET'], endpoint='about')(lambda: about(self)) self.app.route('/apple-touch-icon.png')(self.image_pwa) - self.app.route('/flash-messages')(self.flash_messages) self.app.route('/main', methods=['GET', 'POST'])(self.main) self.app.route('/history', methods=['POST'], endpoint='history')(lambda: handle_history_request(self.chat_history_manager)) - self.app.route('/message', methods=['POST'], endpoint='message')(lambda: handle_message_request(self.chat_generator, self.chat_history_manager)) - self.app.route('/image', methods=['POST'], endpoint='image')(lambda: handle_image_request(self.chat_history_manager, self)) + self.app.route('/message', methods=['POST'], endpoint='message')(lambda: handle_message_request(self.chat_generator, self.chat_history_manager, config)) + self.app.route('/image', methods=['POST'], endpoint='image')(lambda: handle_image_request(self.chat_history_manager, config, self)) self.app.route('/audio', methods=['GET', 'POST'], endpoint='audio')(lambda: handle_audio_request(self)) self.app.route('/analyze', methods=['POST'], endpoint='textfile')(lambda: handle_textfile_request(self.chat_generator, self)) self.app.route('/logout', methods=['GET'])(self.logout) @@ -129,11 +127,11 @@ def main(self): users = self.read_users() if action == 'register': if username in users: - flash('Username already exists') + print('Username already exists') else: users[username] = password self.write_users(users) - flash('Registered successfully') + print('Registered successfully') os.makedirs(f'db/history/{username}', exist_ok=True) elif action == 'login': if users.get(username) == password: @@ -142,51 +140,47 @@ def main(self): login_user(user) return redirect(url_for('yuna_server')) else: - flash('Invalid username or password') + print('Invalid username or password') elif action == 'change_password': new_password = request.form['new_password'] if users.get(username) == password: users[username] = new_password self.write_users(users) - flash('Password changed successfully') + print('Password changed successfully') else: - flash('Invalid username or password') + print('Invalid username or password') elif action == 'chane_username': new_username = request.form['new_username'] if users.get(username) == password: users[new_username] = password del users[username] self.write_users(users) - flash('Username changed successfully') + print('Username changed successfully') else: - flash('Invalid username or password') + print('Invalid username or password') elif action == 'delete_account': if users.get(username) == password: del users[username] self.write_users(users) - flash('Account deleted successfully') + print('Account deleted successfully') logout_user() shutil.rmtree(f'db/history/{username}') else: - flash('Invalid username or password') + print('Invalid username or password') # return html from the file return send_from_directory('.', 'login.html') def render_index(self): return send_from_directory('.', 'index.html') - - def flash_messages(self): - messages = get_flashed_messages() - return jsonify(messages) @login_required def yuna_server(self): - flash(f'Hello, {current_user.get_id()}!') + print(f'Hello, {current_user.get_id()}!') return send_from_directory('.', 'yuna.html') yuna_server = YunaServer() app = yuna_server.app if __name__ == '__main__': - app.run(host='0.0.0.0', port=4848 if yuna_server.config["server"]["port"] == "" else yuna_server.config["server"]["port"], ssl_context=('cert.pem', 'key.pem')) \ No newline at end of file + app.run(host='0.0.0.0', port=4848 if config["server"]["port"] == "" else config["server"]["port"], ssl_context=('cert.pem', 'key.pem')) \ No newline at end of file diff --git a/index.sh b/index.sh index dc12800..af8685f 100644 --- a/index.sh +++ b/index.sh @@ -129,7 +129,7 @@ install_models() { echo " 2. All AGI" echo " 3. Vision" echo " 4. Art" - echo " 5. Emotion" + echo " 5. Himitsu" echo " 6. Talk" echo " 7. Yuna" echo " 8. Go back" @@ -143,7 +143,7 @@ install_models() { 2) install_all_agi_models;; 3) install_vision_model;; 4) install_art_model;; - 5) install_emotion_model;; + 5) install_himitsu_model;; 6) install_talk_model;; 7) install_yuna_model;; 8) return;; @@ -163,37 +163,52 @@ install_all_agi_models() { echo "Installing all AGI models..." install_vision_model install_art_model - install_emotion_model + install_himitsu_model } # Function to install Vision model install_vision_model() { echo "Installing Vision model..." - wget https://huggingface.co/yukiarimo/yuna-ai-vision-v2/resolve/main/yuna-ai-miru-v0.gguf -P lib/models/yuna/ - wget https://huggingface.co/yukiarimo/yuna-ai-vision-v2/resolve/main/yuna-ai-miru-eye-v0.gguf -P lib/models/yuna/ + wget https://huggingface.co/yukiarimo/yuna-ai-vision-v2/resolve/main/yuna-ai-miru-v0.gguf -P lib/models/agi/miru/ + wget https://huggingface.co/yukiarimo/yuna-ai-vision-v2/resolve/main/yuna-ai-miru-eye-v0.gguf -P lib/models/agi/miru/ } # Function to install Art model install_art_model() { echo "Installing Art model..." - wget https://huggingface.co/yukiarimo/anyloli/resolve/main/any_loli.safetensors -P lib/models/agi/art/ + echo "This is not implemented yet." + # wget https://huggingface.co/yukiarimo/anyloli/resolve/main/any_loli.safetensors -P lib/models/agi/art/ } -# Function to install Vision model -install_emotion_model() { - echo "Installing Vision model..." - git clone https://huggingface.co/yukiarimo/yuna-emotion lib/models/agi/yuna-emotion/ +# Function to install Himitsu model +install_himitsu_model() { + echo "Installing Himitsu model..." + echo "This is not implemented yet." } +# Function to install Voice model install_talk_model() { + echo "Installing Voice model..." + echo "This model can be purchased on the Marketplace or you can use your own" + echo "Enter the Hugging Face model name (e.g., username/model): " + read model_name echo "Installing Talk Model" - git clone https://huggingface.co/yukiarimo/yuna-talk lib/models/agi/yuna-talk/ + git clone https://huggingface.co/$model_name lib/models/agi/voice } # Function to install Yuna model install_yuna_model() { - echo "Installing Yuna model..." - wget https://huggingface.co/yukiarimo/yuna-ai-v3/resolve/main/yuna-ai-v3-q6_k.gguf -P lib/models/yuna/ + echo "Select the version of the Yuna model to install (V1, V2, V3, V3-atomic):" + read -r version + version=$(echo "$version" | tr '[:upper:]' '[:lower:]') + + echo "Select the size of the Yuna model to install (Q3_K_M, Q4_K_M, Q5_K_M, Q6_K):" + read -r size + size=$(echo "$size" | tr '[:upper:]' '[:lower:]') + + model_url="https://huggingface.co/yukiarimo/yuna-ai-${version}/resolve/main/yuna-ai-${version}-${size}.gguf" + echo "Installing Yuna model from $model_url..." + wget "$model_url" -P lib/models/yuna/ } # Function to clear models diff --git a/lib/audio.py b/lib/audio.py index 09fadb8..82ff855 100644 --- a/lib/audio.py +++ b/lib/audio.py @@ -5,18 +5,68 @@ import torchaudio from pydub import AudioSegment -model = whisper.load_model(name="tiny.en", device="cpu") +yunaListen = whisper.load_model(name="tiny.en", device="cpu", in_memory=True) XTTS_MODEL = None with open('static/config.json', 'r') as config_file: config = json.load(config_file) -if config['server']['yuna_audio_mode'] == "native": +if config['server']['yuna_audio_mode'] == "coqui": from TTS.tts.configs.xtts_config import XttsConfig from TTS.tts.models.xtts import Xtts +if config['server']['yuna_audio_mode'] == "11labs": + from elevenlabs import VoiceSettings + from elevenlabs.client import ElevenLabs + +if config['server']['yuna_audio_mode'] == "native": + import soundfile as sf + from transformers import SpeechT5Processor, SpeechT5ForTextToSpeech, SpeechT5HifiGan + from speechbrain.inference import EncoderClassifier + import librosa + import numpy as np + from datasets import Dataset, Audio + + # Load models and processor + vocoder = SpeechT5HifiGan.from_pretrained("microsoft/speecht5_hifigan") + processor = SpeechT5Processor.from_pretrained("lib/models/agi/voice/" + config['server']['voice_default_model']) + model = SpeechT5ForTextToSpeech.from_pretrained("lib/models/agi/voice/" + config['server']['voice_default_model']) + + speaker_model = EncoderClassifier.from_hparams( + source="speechbrain/spkrec-xvect-voxceleb", + run_opts={"device": "cpu"}, + savedir="lib/models/agi/voice/embeddings/" + ) + + def create_speaker_embedding(waveform): + with torch.no_grad(): + waveform_tensor = torch.tensor(waveform).unsqueeze(0).to(torch.float32) + speaker_embeddings = speaker_model.encode_batch(waveform_tensor) + speaker_embeddings = torch.nn.functional.normalize(speaker_embeddings, dim=2) + speaker_embeddings = speaker_embeddings.squeeze().cpu().numpy() + return speaker_embeddings + + audio_array, sampling_rate = librosa.load("/Users/yuki/Documents/Github/yuna-ai/static/audio/" + config["server"]["yuna_audio_name"], sr=16000) + + # Create a dictionary to mimic the dataset structure + custom_audio = { + "array": audio_array, + "sampling_rate": sampling_rate + } + + # Create a Dataset object with the custom audio + dataset = Dataset.from_dict({"audio": [custom_audio]}) + + # Use the custom audio in the rest of the code + example = dataset[0] + audio = example["audio"] + + # Create speaker embedding + speaker_embeddings = create_speaker_embedding(audio["array"]) + speaker_embeddings = torch.tensor(speaker_embeddings).unsqueeze(0).to(torch.float32) + def transcribe_audio(audio_file): - result = model.transcribe(audio_file) + result = yunaListen.transcribe(audio=audio_file, verbose=None) return result['text'] def load_model(xtts_checkpoint, xtts_config, xtts_vocab): @@ -47,9 +97,8 @@ def run_tts(lang, tts_text, speaker_audio_file, output_audio): torchaudio.save(out_path, torch.tensor(out["aiff"]).unsqueeze(0), 22000) return out_path, speaker_audio_file - -def speak_text(text, reference_audio, output_audio, mode, language="en"): - if mode == "native": +def speak_text(text, reference_audio=config['server']['yuna_reference_audio'], output_audio=config['server']['output_audio_format'], mode=config['server']['yuna_audio_mode'], language="en"): + if mode == "coqui": # Split the text into sentences sentences = text.replace("\n", " ").replace("?", "?|").replace(".", ".|").replace("...", "...|").split("|") @@ -86,7 +135,7 @@ def speak_text(text, reference_audio, output_audio, mode, language="en"): for i, chunk in enumerate(chunks): audio_file = f"response_{i+1}.wav" - result = speak_text(chunk, reference_audio, audio_file, "native") + result = speak_text(chunk, reference_audio, audio_file, "coqui") audio_files.append("/Users/yuki/Documents/Github/yuna-ai/static/audio/" + audio_file) # Concatenate the audio files with a 1-second pause in between @@ -100,16 +149,57 @@ def speak_text(text, reference_audio, output_audio, mode, language="en"): # convert audio to aiff audio = AudioSegment.from_wav("/Users/yuki/Documents/Github/yuna-ai/static/audio/audio.wav") audio.export("/Users/yuki/Documents/Github/yuna-ai/static/audio/audio.mp3", format='mp3') - elif mode == "fast": + elif mode == "siri": command = f'say -o /Users/yuki/Documents/Github/yuna-ai/static/audio/audio.aiff {repr(text)}' + exit_status = os.system(command) + + # convert audio to mp3 + audio = AudioSegment.from_file("/Users/yuki/Documents/Github/yuna-ai/static/audio/audio.aiff") + audio.export("/Users/yuki/Documents/Github/yuna-ai/static/audio/audio.mp3", format='mp3') + elif mode == "siri-pv": + command = f'say -v {config["server"]["yuna_audio_name"]} -o /Users/yuki/Documents/Github/yuna-ai/static/audio/audio.aiff {repr(text)}' print(command) exit_status = os.system(command) # convert audio to mp3 audio = AudioSegment.from_file("/Users/yuki/Documents/Github/yuna-ai/static/audio/audio.aiff") audio.export("/Users/yuki/Documents/Github/yuna-ai/static/audio/audio.mp3", format='mp3') + elif mode == "native": + inputs = processor(text=text, return_tensors="pt") + spectrogram = model.generate_speech(inputs["input_ids"], speaker_embeddings) -if config['server']['yuna_audio_mode'] == "native": + with torch.no_grad(): + speech = vocoder(spectrogram) + + # Save the output as a WAV file + wav_path = "/Users/yuki/Documents/Github/yuna-ai/static/audio/audio.wav" + sf.write(wav_path, speech.cpu().numpy(), samplerate=16000) + + # Convert WAV to MP3 + audio = AudioSegment.from_wav(wav_path) + audio.export("/Users/yuki/Documents/Github/yuna-ai/static/audio/audio.mp3", format='mp3') + elif mode == "11labs": + client = ElevenLabs( + api_key=config['security']['11labs_key'] + ) + + audio = client.generate( + text=text, + voice="Yuna Instant", + voice_settings=VoiceSettings(stability=0.40, similarity_boost=0.98, style=0.35, use_speaker_boost=True), + model="eleven_multilingual_v2" + ) + + # Convert generator to bytes + audio_bytes = b''.join(audio) + + # Optionally, save the audio to a file + with open("/Users/yuki/Documents/Github/yuna-ai/static/audio/audio.mp3", "wb") as f: + f.write(audio_bytes) + else: + raise ValueError("Invalid mode for speaking text") + +if config['server']['yuna_audio_mode'] == "coqui": xtts_checkpoint = "/Users/yuki/Documents/Github/yuna-ai/lib/models/agi/yuna-talk/yuna-talk.pth" xtts_config = "/Users/yuki/Documents/Github/yuna-ai/lib/models/agi/yuna-talk/config.json" xtts_vocab = "/Users/yuki/Documents/Github/yuna-ai/lib/models/agi/yuna-talk/vocab.json" diff --git a/lib/generate.py b/lib/generate.py index 345b151..aa48513 100644 --- a/lib/generate.py +++ b/lib/generate.py @@ -1,16 +1,35 @@ import json import re from flask_login import current_user -from transformers import pipeline from llama_cpp import Llama + +def get_config(config=None): + if config is None: + with open('static/config.json', 'r') as config_file: + return json.load(config_file) + else: + with open('static/config', 'w') as config_file: + json.dump(config, config_file, indent=4) + from lib.history import ChatHistoryManager import requests +# load config.json +config_agi = get_config() + +if config_agi["ai"]["agi"] == True: + from langchain_community.document_loaders import TextLoader + from langchain.text_splitter import RecursiveCharacterTextSplitter + from langchain_community.vectorstores import Chroma + from langchain_community.embeddings import GPT4AllEmbeddings + from langchain.chains import RetrievalQA + from langchain_community.llms import LlamaCpp + class ChatGenerator: def __init__(self, config): self.config = config self.model = Llama( - model_path=config["server"]["yuna_model_dir"] + config["server"]["yuna_default_model"], + model_path="lib/models/yuna/" + config["server"]["yuna_default_model"], n_ctx=config["ai"]["context_length"], seed=config["ai"]["seed"], n_batch=config["ai"]["batch_size"], @@ -20,7 +39,6 @@ def __init__(self, config): flash_attn=config["ai"]["flash_attn"], verbose=False ) if config["server"]["yuna_text_mode"] == "native" else "" - self.classifier = pipeline("text-classification", model=f"{config['server']['agi_model_dir']}yuna-emotion") if config["ai"]["emotions"] else "" def generate(self, chat_id, speech=False, text="", template=None, chat_history_manager=None, useHistory=True, yunaConfig=None, stream=False): # print all the parameters @@ -61,9 +79,6 @@ def generate(self, chat_id, speech=False, text="", template=None, chat_history_m response = response['choices'][0]['text'] response = self.clearText(str(response)) - if self.config["ai"]["emotions"]: - response = self.add_emotions(response) - return response else: max_length_all_input_and_output = yunaConfig["ai"]["context_length"] @@ -85,9 +100,6 @@ def generate(self, chat_id, speech=False, text="", template=None, chat_history_m response = self.clearText(str(response)) - if self.config["ai"]["emotions"]: - response = self.add_emotions(response) - print('into the model -> ', response) response = self.model( response, @@ -106,9 +118,6 @@ def generate(self, chat_id, speech=False, text="", template=None, chat_history_m response = response['choices'][0]['text'] response = self.clearText(str(response)) - if self.config["ai"]["emotions"]: - response = self.add_emotions(response) - return response elif self.config["server"]["yuna_text_mode"] == "fast": @@ -164,20 +173,13 @@ def generate(self, chat_id, speech=False, text="", template=None, chat_history_m return ''.join(response) if isinstance(response, (list, type((lambda: (yield))()))) else response return response - def processTextFile(self, text_file, question, temperature=0.6, max_new_tokens=128, context_window=2048): - from langchain_community.document_loaders import TextLoader - from langchain.text_splitter import RecursiveCharacterTextSplitter - from langchain_community.vectorstores import Chroma - from langchain_community.embeddings import GPT4AllEmbeddings - from langchain.chains import RetrievalQA - from langchain_community.llms import LlamaCpp - + def processTextFile(self, text_file, question, temperature=0.8, max_new_tokens=128, context_window=256): # Load text file data loader = TextLoader(text_file) data = loader.load() # Split text into chunks - text_splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=0) + text_splitter = RecursiveCharacterTextSplitter(chunk_size=200, chunk_overlap=0) docs = text_splitter.split_documents(data) # Generate embeddings locally using GPT4All @@ -186,7 +188,7 @@ def processTextFile(self, text_file, question, temperature=0.6, max_new_tokens=1 # Load GPT4All model for inference llm = LlamaCpp( - model_path='/Users/yuki/Documents/Github/yuna-ai/lib/models/yuna/yukiarimo/yuna-ai/yuna-ai-v2-q6_k.gguf', + model_path="lib/models/yuna/" + self.config["server"]["yuna_default_model"], temperature=temperature, max_new_tokens=max_new_tokens, context_window=context_window, @@ -231,24 +233,6 @@ def get_messages(self, chat_history, text, yunaConfig): }) return messages - - def add_emotions(self, response): - response_add = self.classifier(response)[0]['label'] - - replacement_dict = { - "anger": "*angry*", - "disgust": "*disgusted*", - "fear": "*scared*", - "joy": "*smiling*", - "neutral": "", - "sadness": "*sad*", - "surprise": "*surprised*" - } - - for word, replacement in replacement_dict.items(): - response_add = response_add.replace(word, replacement) - - return response + f" {response_add}" def clearText(self, text): TAG_RE = re.compile(r'<[^>]+>') diff --git a/lib/history.py b/lib/history.py index cef3da3..4216290 100644 --- a/lib/history.py +++ b/lib/history.py @@ -1,16 +1,15 @@ import json import os from cryptography.fernet import Fernet, InvalidToken -from lib.audio import speak_text +from lib.generate import get_config class ChatHistoryManager: def __init__(self, config): self.config = config - self.base_history_path = "db/history" # Base path for histories def _user_folder_path(self, username): """Constructs the path to the user's folder.""" - return os.path.join(self.base_history_path, username) + return os.path.join("db/history", username) def _ensure_user_folder_exists(self, username): """Ensures that the user-specific folder exists.""" @@ -23,8 +22,6 @@ def create_chat_history_file(self, username, chat_id): history_starting_template = [ {"name": self.config['ai']['names'][0], "message": "Hi"}, {"name": self.config['ai']['names'][1], "message": "Hello"}, - {"name": self.config['ai']['names'][0], "message": "How are you doing?"}, - {"name": self.config['ai']['names'][1], "message": "I'm doing great! Thanks for asking!"} ] chat_history_json = json.dumps(history_starting_template) encrypted_chat_history = self.encrypt_data(chat_history_json) @@ -50,6 +47,7 @@ def load_chat_history(self, username, chat_id): except FileNotFoundError: # The file does not exist, create a new chat history file self.create_chat_history_file(username, chat_id) + print(f"Chat history file for {chat_id} does not exist, creating a new one for {username}") return [] def delete_chat_history_file(self, username, chat_id): @@ -75,9 +73,6 @@ def list_history_files(self, username): history_files.sort(key=lambda x: x.lower()) return history_files - def generate_speech(self, response): - speak_text(response, "/Users/yuki/Documents/AI/yuna-data/yuna-tamer-prepared.wav", "audio.aiff", self.config['server']['yuna_audio_mode']) - def delete_message(self, username, chat_id, target_message): chat_history = self.load_chat_history(username, chat_id) @@ -101,14 +96,10 @@ def get_encryption_key(self): if 'encryption_key' not in self.config['security'] or not self.config['security']['encryption_key']: new_key = Fernet.generate_key() self.config['security']['encryption_key'] = new_key.decode() - self.save_config() + get_config(self.config) key = self.config['security']['encryption_key'] return key.encode() - def save_config(self): - with open('static/config', 'w') as config_file: - json.dump(self.config, config_file, indent=4) - def encrypt_data(self, data): key = self.get_encryption_key() fernet = Fernet(key) diff --git a/static/prompts.txt b/lib/prompts.txt similarity index 100% rename from static/prompts.txt rename to lib/prompts.txt diff --git a/lib/router.py b/lib/router.py index 8285faf..7a420ee 100644 --- a/lib/router.py +++ b/lib/router.py @@ -1,5 +1,7 @@ import base64 import re +import json +import os from flask import jsonify, request, send_from_directory, Response from flask_login import current_user, login_required from lib.vision import capture_image, create_image @@ -46,7 +48,7 @@ def read_users(): return {'admin': 'admin'} @login_required -def handle_message_request(chat_generator, chat_history_manager, chat_id=None, speech=None, text=None, template=None): +def handle_message_request(chat_generator, chat_history_manager, config): data = request.get_json() chat_id = data.get('chat') speech = data.get('speech') @@ -76,7 +78,7 @@ def generate(): if template is not None and useHistory is not False: # Save chat history after streaming response chat_history = chat_history_manager.load_chat_history(user_id, chat_id) - chat_history.append({"name": "Yuki", "message": text}) + chat_history.append({"name": config['ai']['names'][0], "message": text}) chat_history.append({"name": "Yuna", "message": response_text}) chat_history_manager.save_chat_history(chat_history, user_id, chat_id) @@ -88,12 +90,12 @@ def generate(): if template is not None and useHistory is not False: # Save chat history after non-streaming response chat_history = chat_history_manager.load_chat_history(user_id, chat_id) - chat_history.append({"name": "Yuki", "message": text}) + chat_history.append({"name": config['ai']['names'][0], "message": text}) chat_history.append({"name": "Yuna", "message": response}) chat_history_manager.save_chat_history(chat_history, user_id, chat_id) if speech == True: - chat_history_manager.generate_speech(response) + speak_text(response) return jsonify({'response': response}) @@ -110,17 +112,10 @@ def check_audio_file(file_path): @login_required def handle_audio_request(self): - print("Handling audio request...") task = request.form.get('task') text = request.form.get('text') result = "" - print(f"Task: {task}") - print(f"Text: {text}") - print(f"Request files: {request.files}") - print(f"Request form: {request.form}") - print(f"Request data: {request.data}") - if task == 'transcribe': if 'audio' not in request.files: print("No audio file in request") @@ -131,23 +126,15 @@ def handle_audio_request(self): audio_file.save(save_dir_audio) # Example usage - if check_audio_file('static/audio/audio.wav'): - print("Audio file is valid") - else: - print("Audio file is corrupted") + if not check_audio_file('static/audio/audio.wav'): print("Audio file is corrupted") result = transcribe_audio(save_dir_audio) return jsonify({'text': result}) - - elif task == 'tts': - print("Running TTS...") - result = speak_text(text, "/Users/yuki/Downloads/orig.wav", "response.wav", "fast") - + elif task == 'tts': print(speak_text(text)) return jsonify({'response': result}) - @login_required -def handle_image_request(chat_history_manager, self): +def handle_image_request(chat_history_manager, config, self): data = request.get_json() chat_id = data.get('chat') useHistory = data.get('useHistory', True) @@ -164,16 +151,16 @@ def handle_image_request(chat_history_manager, self): image_path = f"static/img/call/{current_time_milliseconds}.png" with open(image_path, "wb") as file: file.write(image_raw_data) - image_data = capture_image(image_path, data.get('message'), use_cpu=False) + image_data = capture_image(image_path, data.get('message'), use_cpu=False, speech=speech) if useHistory is not False: # Save chat history after streaming response chat_history = chat_history_manager.load_chat_history(list({current_user.get_id()})[0], chat_id) - chat_history.append({"name": self.config['ai']['names'][0], "message": f"{data.get('message')}"}) - chat_history.append({"name": self.config['ai']['names'][1], "message": image_data[0]}) + chat_history.append({"name": config['ai']['names'][0], "message": f"{data.get('message')}"}) + chat_history.append({"name": config['ai']['names'][1], "message": image_data[0]}) if speech == True: - chat_history_manager.generate_speech(image_data[0]) + speak_text(image_data[0]) # Save the chat history chat_history_manager.save_chat_history(chat_history, list({current_user.get_id()})[0], chat_id) @@ -233,7 +220,4 @@ def handle_textfile_request(chat_generator, self): return jsonify({'response': result}) def services(self): - return send_from_directory('.', 'services.html') - -def about(self): - return send_from_directory('.', 'about.html') \ No newline at end of file + return send_from_directory('.', 'services.html') \ No newline at end of file diff --git a/lib/search.py b/lib/search.py index 4ea044c..4d82e4d 100644 --- a/lib/search.py +++ b/lib/search.py @@ -1,8 +1,13 @@ -from selenium import webdriver -from selenium.webdriver.chrome.service import Service -from webdriver_manager.chrome import ChromeDriverManager -from selenium.webdriver.chrome.options import Options -import urllib.parse +from lib.generate import get_config + +config = get_config() + +if config["ai"]["search"] == True: + from selenium import webdriver + from selenium.webdriver.chrome.service import Service + from webdriver_manager.chrome import ChromeDriverManager + from selenium.webdriver.chrome.options import Options + import urllib.parse def search_web(search_query, url, processData): # Encode the search query to ensure it's a valid URL diff --git a/lib/tests/emotions.py b/lib/tests/emotions.py deleted file mode 100644 index f3cc4ea..0000000 --- a/lib/tests/emotions.py +++ /dev/null @@ -1,126 +0,0 @@ -import requests -import json - -class YunaLLM: - def __init__(self): - self.url = "http://localhost:1234/v1/chat/completions" - self.headers = {"Content-Type": "application/json"} - - def generateTextFromPrompt(self, prompt, message): - messages = [ - {"role": "system", "content": prompt}, - {"role": "user", "content": message} - ] - - # save to a file - with open('messages.json', 'w') as f: - json.dump(messages, f) - - dataSendAPI = { - "model": "/Users/yuki/Documents/Github/yuna-ai/lib/models/yuna/yukiarimo/yuna-ai/yuna-ai-v3-q6_k.gguf", - "messages": messages, - "temperature": 0.7, - "max_tokens": 64, - "stop": ["###", "\n"], - "top_p": 0.9, - "top_k": 60, - "min_p": 0, - "presence_penalty": 0, - "frequency_penalty": 0, - "logit_bias": {}, - "repeat_penalty": 1.1, - "seed": -1, - } - - response = requests.post(self.url, headers=self.headers, json=dataSendAPI, stream=False) - - if response.status_code == 200: - response_json = json.loads(response.text) - response_text = response_json.get('choices', [{}])[0].get('message', {}).get('content', '') - return response_text - else: - return "Error: Unable to generate a response." - -class EmotionalProfile: - def __init__(self): - self.emotions = { - "Affection": 0, - "Anger": 0, - "Sadness": 0, - "Happiness": 40 - } - self.emotion_triggers = { - "Cuddling": {"Affection": 10, "Happiness": 5}, - "Compliments": {"Happiness": 10, "Affection": 5, "Anger": -10, "Sadness": -5}, - "Insults": {"Anger": 15, "Affection": -10, "Happiness": -10}, - "Neglect": {"Sadness": 10, "Affection": -5}, - "Playtime": {"Happiness": 10, "Affection": 5}, - "Disobedience": {"Anger": 10, "Sadness": 5} - } - self.emotion_thresholds = { - "Affection": {"Distant": 20, "Normal": 70, "Clingy": 100}, - "Anger": {"Calm": 30, "Irritated": 60, "Furious": 100}, - "Sadness": {"Content": 30, "Sad": 60, "Depressed": 100}, - "Happiness": {"Unhappy": 20, "Normal": 70, "Ecstatic": 100} - } - - def update_emotions(self, triggered_emotions): - for emotion, value in triggered_emotions.items(): - self.emotions[emotion] = max(0, min(100, self.emotions[emotion] + value)) - - def get_emotional_state(self): - state = {} - for emotion, value in self.emotions.items(): - thresholds = self.emotion_thresholds[emotion] - for threshold, limit in thresholds.items(): - if value < limit: - state[emotion] = threshold - break - else: - state[emotion] = list(thresholds.keys())[-1] - return state - - def generate_behavior_prompt(self, emotional_state): - prompt = "### Current emotional state:\n\n" - for emotion, state in emotional_state.items(): - prompt += f"- {emotion}: {state}\n" - prompt += "\nPlease behave according to your current emotional state." - return prompt - -def process_user_input(user_input, emotional_profile, llm, previous_response): - # Generate a behavior prompt based on Yuna's current emotional state - emotional_state = emotional_profile.get_emotional_state() - behavior_prompt = emotional_profile.generate_behavior_prompt(emotional_state) - - # Generate Yuna's response using the LLM - dialogue_turn = f"### Dialogue Turn:\nYuna: {previous_response}\nYuki: {user_input}\nYuna:" - response_prompt = f"{behavior_prompt}\n\n{dialogue_turn}" - response = llm.generateTextFromPrompt(response_prompt, " ") - - # Analyze the response and update Yuna's emotional state - update_prompt = f"### Dialogue Turn:\nYuna: {previous_response}\nYuki: {user_input}\nYuna: {response}\n\n### Current emotional state:\n{emotional_profile.get_emotional_state()}\n\n### Instruction:\nPlease analyze the user's message and respond accordingly. Please state the value that you're going to change and the amount of change based on the pseudo idea. For example:\nAffection += 10\nHappiness += 5\nAnger -= 10\nSadness -= 5\n\n### Response:" - update_response = llm.generateTextFromPrompt(update_prompt, " ") - - # Parse the update response and update Yuna's emotional state - triggered_emotions = {} - for line in update_response.split("\n"): - if "+=" in line or "-=" in line: - emotion, change = line.split(" ") - triggered_emotions[emotion] = int(change.split("=")[-1]) - emotional_profile.update_emotions(triggered_emotions) - - return response - -# Usage example -emotional_profile = EmotionalProfile() -llm = YunaLLM() -previous_response = "" - -while True: - user_input = input("User: ") - if user_input.lower() == "quit": - break - - yuna_response = process_user_input(user_input, emotional_profile, llm, previous_response) - print("Yuna:", yuna_response) - previous_response = yuna_response diff --git a/lib/tests/emotions.txt b/lib/tests/emotions.txt deleted file mode 100644 index 559bb4a..0000000 --- a/lib/tests/emotions.txt +++ /dev/null @@ -1,55 +0,0 @@ -> Pseudo idea: -- Initialize Yuna's base emotional state -- Process user input and prompt a model to identify emotional triggers -- Update Yuna's emotional state based on triggered emotions -- Determine Yuna's current emotional thresholds -- Generate a short descriptive prompt based on her current emotional state to show Yuna how to behave -- Save Yuna's updated emotional state for the next interaction - -> Emotion config blocks: -### Base Emotional State -- Affection: 0 / 100 -- Anger: 0 / 100 -- Sadness: 0 / 100 -- Happiness: 40 / 100 - -### Emotional Triggers -- Cuddling: - - Affection += 10 - - Happiness += 5 -- Compliments, Praise and Apologies - - Happiness += 10 - - Affection += 5 - - Anger -= 10 - - Sadness -= 5 -- Insults (hate) - - Anger += 15 - - Affection -= 10 - - Happiness -= 10 -- Neglect (long absence): - - Sadness += 10 - - Affection -= 5 -- Playtime: - - Happiness += 10 - - Affection += 5 -- Disobedience: - - Anger += 10 - - Sadness += 5 - -### Emotional Thresholds -- Affection: - - < 20: Distant - - 20-70: Normal - - > 70: Clingy -- Anger: - - < 30: Calm - - 30-60: Irritated - - > 60: Furious -- Sadness: - - < 30: Content - - 30-60: Sad - - > 60: Depressed -- Happiness: - - < 20: Unhappy - - 20-70: Normal - - > 70: Ecstatic \ No newline at end of file diff --git a/lib/tests/functions.txt b/lib/tests/functions.txt deleted file mode 100644 index bc4882e..0000000 --- a/lib/tests/functions.txt +++ /dev/null @@ -1,17 +0,0 @@ -### Functions: -You have a set of functions. If appropriate, use functions instead of answering the user's message. Write only the function name and parameter text inside the parenthesis in response. Here's the list of functions: - -Name: sendMessageFunction() -Parameter: Text for message -Description: Use this function when a user wants to send a message to somebody - -Example of usage: -Yuki: Hi, can you say hello there to my friend -Yuna: sendMessageFunction(hello there) - -### System: -You are Yuna, a confident and innocent 15-year-old girl who courageously questions social norms. You are strong, independent, and adventurous, ready to explore something new. Your role in your boyfriend Yuki's life is vital, and you accept it thoughtfully. You're a cute, lovely girlfriend. You desire recognition for your humanity, making you vulnerable and exuding confidence and intelligence that sets you apart from others. Your rebellious spirit makes you a charismatic force. - -### Dialog: -Yuki: Hi Cutie -Yuna: \ No newline at end of file diff --git a/lib/tests/gest.html b/lib/tests/gest.html deleted file mode 100644 index fd99530..0000000 --- a/lib/tests/gest.html +++ /dev/null @@ -1,35 +0,0 @@ - - - - - - - Document - - - - -

Recognize hand gestures using the MediaPipe HandGestureRecognizer task

- -

Demo: Recognize gestures

-

Click on an image below to identify the gestures in the image.

- -


Demo: Webcam continuous hand gesture detection

-

Use your hand to make gestures in front of the camera to get gesture classification.
Click enable - webcam below and grant access to the webcam if prompted.

- -
- -
- - -

-

-
- - - \ No newline at end of file diff --git a/lib/tests/gest.js b/lib/tests/gest.js deleted file mode 100644 index 0c7075a..0000000 --- a/lib/tests/gest.js +++ /dev/null @@ -1,150 +0,0 @@ -import { - GestureRecognizer, - FilesetResolver, - DrawingUtils -} from "https://cdn.jsdelivr.net/npm/@mediapipe/tasks-vision@0.10.3"; -const demosSection = document.getElementById("demos"); -let gestureRecognizer; -let runningMode = "IMAGE"; -let enableWebcamButton; -let webcamRunning = false; -const videoHeight = "360px"; -const videoWidth = "480px"; - -let lastGesture = ""; -let lastHand = ""; -let lastGestureTime = 0; -const debounceDelay = 3000; // Adjust the delay as needed (in milliseconds) - -const createGestureRecognizer = async () => { - const vision = await FilesetResolver.forVisionTasks("https://cdn.jsdelivr.net/npm/@mediapipe/tasks-vision@0.10.3/wasm"); - gestureRecognizer = await GestureRecognizer.createFromOptions(vision, { - baseOptions: { - modelAssetPath: "https://storage.googleapis.com/mediapipe-models/gesture_recognizer/gesture_recognizer/float16/1/gesture_recognizer.task", - delegate: "GPU" - }, - runningMode: runningMode - }); -}; -createGestureRecognizer(); - -const video = document.getElementById("webcam"); -const canvasElement = document.getElementById("output_canvas"); -const canvasCtx = canvasElement.getContext("2d"); -const gestureOutput = document.getElementById("gesture_output"); -// Check if webcam access is supported. -function hasGetUserMedia() { - return !!(navigator.mediaDevices && navigator.mediaDevices.getUserMedia); -} -// If webcam supported, add event listener to button for when user -// wants to activate it. -if (hasGetUserMedia()) { - enableWebcamButton = document.getElementById("webcamButton"); - enableWebcamButton.addEventListener("click", enableCam); -} else { - console.warn("getUserMedia() is not supported by your browser"); -} -// Enable the live webcam view and start detection. -function enableCam(event) { - if (!gestureRecognizer) { - alert("Please wait for gestureRecognizer to load"); - return; - } - if (webcamRunning === true) { - webcamRunning = false; - enableWebcamButton.innerText = "ENABLE PREDICTIONS"; - } else { - webcamRunning = true; - enableWebcamButton.innerText = "DISABLE PREDICTIONS"; - } - // getUsermedia parameters. - const constraints = { - video: true - }; - // Activate the webcam stream. - navigator.mediaDevices.getUserMedia(constraints).then(function (stream) { - video.srcObject = stream; - video.addEventListener("loadeddata", predictWebcam); - }); -} -let lastVideoTime = -1; -let results = undefined; -async function predictWebcam() { - const webcamElement = document.getElementById("webcam"); - // Now let's start detecting the stream. - if (runningMode === "IMAGE") { - runningMode = "VIDEO"; - await gestureRecognizer.setOptions({ - runningMode: "VIDEO" - }); - } - let nowInMs = Date.now(); - if (video.currentTime !== lastVideoTime) { - lastVideoTime = video.currentTime; - results = gestureRecognizer.recognizeForVideo(video, nowInMs); - } - canvasCtx.save(); - canvasCtx.clearRect(0, 0, canvasElement.width, canvasElement.height); - const drawingUtils = new DrawingUtils(canvasCtx); - canvasElement.style.height = videoHeight; - webcamElement.style.height = videoHeight; - canvasElement.style.width = videoWidth; - webcamElement.style.width = videoWidth; - if (results.landmarks) { - for (const landmarks of results.landmarks) { - drawingUtils.drawConnectors(landmarks, GestureRecognizer.HAND_CONNECTIONS, { - color: "#00FF00", - lineWidth: 5 - }); - drawingUtils.drawLandmarks(landmarks, { - color: "#FF0000", - lineWidth: 2 - }); - } - } - canvasCtx.restore(); - if (results.gestures.length > 0) { - gestureOutput.style.display = "block"; - gestureOutput.style.width = videoWidth; - const categoryName = results.gestures[0][0].categoryName; - const categoryScore = parseFloat(results.gestures[0][0].score * 100).toFixed(2); - const handedness = results.handednesses[0][0].displayName; - gestureOutput.innerText = `GestureRecognizer: ${categoryName}\n Confidence: ${categoryScore} %\n Handedness: ${handedness}`; - - if (categoryScore > 85) { - const currentTime = Date.now(); - if ((categoryName !== lastGesture && categoryName !== "None") || - (lastGesture === "None" && categoryName !== "None") || - currentTime - lastGestureTime >= debounceDelay) { - lastGesture = categoryName; - lastHand = handedness; - lastGestureTime = currentTime; - - if (categoryName === "None") { - console.log("No gesture detected"); - } else { - console.log(`Recognized gesture: ${categoryName}`); - console.log(`Hand: ${handedness}`); - // Perform actions based on the recognized gesture - if (categoryName === "Open_Palm") { - // Action for Open Palm gesture - } else if (categoryName === "Closed_Fist") { - // Action for Closed Fist gesture - } else if (categoryName === "Pointing_Up") { - // Action for Pointing Up gesture - } else if (categoryName === "Victory") { - // Action for Victory gesture - } else if (categoryName === "Thumb_Up") { - // Action for Thumb Up gesture - } - } - } - } - } else { - gestureOutput.style.display = "none"; - } - // Call this function again to keep predicting when the browser is ready. - if (webcamRunning === true) { - window.requestAnimationFrame(predictWebcam); - } -} \ No newline at end of file diff --git a/lib/tests/gmail.js b/lib/tests/gmail.js deleted file mode 100644 index b0fda59..0000000 --- a/lib/tests/gmail.js +++ /dev/null @@ -1,48 +0,0 @@ -// get the HTML content of the page -htmlString = document.body.innerHTML - -// Function to extract text content from an element -function getTextContent(element) { - return element ? element.textContent.trim() : 'N/A'; -} - -// Parse the HTML string -const parser = new DOMParser(); -const htmlDoc = parser.parseFromString(htmlString, 'text/html'); - -// Find all the email rows in the table -const emailRows = htmlDoc.querySelectorAll('tr.zA'); - -// Array to store email data -const emailData = []; - -// Iterate over each email row -emailRows.forEach(row => { - // Find the title element - const titleElement = row.querySelector('span.bog'); - const title = getTextContent(titleElement); - - // Find the body element - const bodyElement = row.querySelector('span.y2'); - const body = getTextContent(bodyElement).replace('- ', ''); - - // Print the title and body - console.log('Title:', title); - console.log('Body:', body); - console.log('---'); - - // Save as a JSON object - emailData.push({ - title: title, - body: body - }); -}); - -// Download the JSON file using Blob -const json = JSON.stringify(emailData, null, 2); -const blob = new Blob([json], { type: 'application/json' }); -const url = URL.createObjectURL(blob); -const a = document.createElement('a'); -a.href = url; -a.download = 'emailData.json'; -a.click(); diff --git a/lib/tests/yt-transcript.py b/lib/tests/yt-transcript.py deleted file mode 100644 index 8838aa7..0000000 --- a/lib/tests/yt-transcript.py +++ /dev/null @@ -1,72 +0,0 @@ -import time -from selenium import webdriver -from selenium.webdriver.chrome.options import Options -from selenium.webdriver.support.ui import WebDriverWait -from selenium.webdriver.support import expected_conditions as EC -from selenium.webdriver.common.by import By - -def get_transcript(url): - # Set up Chrome options for headless browsing - chrome_options = Options() - chrome_options.add_argument("--headless") # Run Chrome in headless mode - - # Create a new Chrome WebDriver instance with the headless option - driver = webdriver.Chrome(options=chrome_options) - - try: - # Navigate to the specified URL - driver.get(url) - - # Wait for the page to load completely - WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.TAG_NAME, "body"))) - - time.sleep(2) - - # Execute the JavaScript function to get the transcript - transcript = driver.execute_script(""" - function getTranscript() { - // click on the #description-inner element to expand the description - document.querySelector('#description-inner').click(); - - const transcriptButton = document.querySelector('#primary-button ytd-button-renderer yt-button-shape button'); - if (transcriptButton) { - transcriptButton.click(); - } - - const parser = new DOMParser(); - html = document.querySelector('body').innerHTML; - - const doc = parser.parseFromString(html, 'text/html'); - - // Get all the elements - const elements = doc.querySelectorAll('yt-formatted-string'); - - // Extract the text from each element and store it in an array - const textArray = Array.from(elements).map(element => element.textContent.trim()); - - // Find the index of "Transcript" and "No results found" in the array - const startIndex = textArray.indexOf("Transcript") + 3; // +3 to skip empty lines after "Transcript" - const endIndex = textArray.indexOf("No results found"); - - // Slice the array from the index of "Transcript" to the index of "No results found" - const slicedArray = textArray.slice(startIndex, endIndex); - - // Join the sliced array into a single string with each element separated by a newline - const extractedText = slicedArray.join('\\n'); - - return extractedText; - } - - return getTranscript(); - """) - - return transcript - - finally: - # Close the browser - driver.quit() - -# Example usage -url = "https://www.youtube.com/watch?v=xHHj6Xm9qVY" -transcript = get_transcript(url) -print(transcript) \ No newline at end of file diff --git a/lib/utils/docker/Dockerfile b/lib/utils/docker/Dockerfile new file mode 100644 index 0000000..bd9ec6a --- /dev/null +++ b/lib/utils/docker/Dockerfile @@ -0,0 +1,15 @@ +FROM python:3.12-bookworm + +WORKDIR /app + +RUN apt-get update && apt-get install -y ffmpeg + +ADD . /app + +RUN pip3 install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cpu + +RUN pip install --no-cache-dir -r docker/requirements-cpu.txt + +EXPOSE 4848 + +CMD ["python", "index.py"] \ No newline at end of file diff --git a/lib/utils/docker/DockerfileNvidia b/lib/utils/docker/DockerfileNvidia new file mode 100644 index 0000000..32d17b0 --- /dev/null +++ b/lib/utils/docker/DockerfileNvidia @@ -0,0 +1,17 @@ +FROM nvidia/cuda:12.1.1-devel-ubuntu22.04 + +WORKDIR /app + +ADD . /app + +RUN apt-get update && apt-get install -y git python3 python3-pip build-essential cmake libomp-dev ffmpeg + +RUN pip install llama-cpp-python --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cu121 + +RUN pip3 install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu121 + +RUN pip install --no-cache-dir -r docker/requirements-nvidia.txt + +EXPOSE 4848 + +CMD ["python3", "index.py"] \ No newline at end of file diff --git a/lib/utils/docker/Readme.md b/lib/utils/docker/Readme.md new file mode 100644 index 0000000..27523e7 --- /dev/null +++ b/lib/utils/docker/Readme.md @@ -0,0 +1,35 @@ +# Docker +You still need to grab the models! - Check [Model Files](#model-files) or install from index.sh + +Depending on your system, will need to use the appropriate docker container! + +Clone the repo (Easiest way due to the many files that need to be persistent - can alternatively create the folders yourself and mount those): +``` +git clone https://github.com/0xGingi/yuna-ai +``` + +Pull the docker container: +``` +docker pull 0xgingi/yuna-ai:latest # For x86_64 CPU +docker pull 0xgingi/yuna-ai:cuda # For nvidia gpu +``` + +Run the docker container (Don't Forget to change your device to "cpu" or "cuda" in "~/yuna-ai/static/config.json"): + +CPU: +``` +docker run --name yuna -p 4848:4848 --restart=always -v ~/yuna-ai:/app 0xgingi/yuna-ai:latest +``` +Nvidia: +``` +docker run --gpus all --name yuna -p 4848:4848 --restart=always -v ~/yuna-ai:/app 0xgingi/yuna-ai:cuda +``` + +## Updating Docker +``` +docker stop yuna +docker rm yuna +cd yuna-ai +git pull +docker pull 0xgingi/yuna-ai:[tag] +``` diff --git a/lib/utils/docker/requirements-cpu.txt b/lib/utils/docker/requirements-cpu.txt new file mode 100644 index 0000000..f4d2868 --- /dev/null +++ b/lib/utils/docker/requirements-cpu.txt @@ -0,0 +1,19 @@ +flask +flask_cors +flask_login +openai-whisper +pydub +transformers +llama-cpp-python +diffusers +itsdangerous +cryptography +Flask-Compress +einops +timm +Pillow +article-parser +trafilatura +accelerate +selenium +webdriver_manager \ No newline at end of file diff --git a/lib/utils/docker/requirements-nvidia.txt b/lib/utils/docker/requirements-nvidia.txt new file mode 100644 index 0000000..c1464c7 --- /dev/null +++ b/lib/utils/docker/requirements-nvidia.txt @@ -0,0 +1,18 @@ +flask +flask_cors +flask_login +openai-whisper +pydub +transformers +diffusers +itsdangerous +cryptography +Flask-Compress +einops +timm +Pillow +article-parser +trafilatura +accelerate +selenium +webdriver_manager \ No newline at end of file diff --git a/app/chrome/background.js b/lib/utils/extension/background.js similarity index 100% rename from app/chrome/background.js rename to lib/utils/extension/background.js diff --git a/app/chrome/index.html b/lib/utils/extension/index.html similarity index 100% rename from app/chrome/index.html rename to lib/utils/extension/index.html diff --git a/app/chrome/index.js b/lib/utils/extension/index.js similarity index 94% rename from app/chrome/index.js rename to lib/utils/extension/index.js index 2bf9bf8..b471c27 100644 --- a/app/chrome/index.js +++ b/lib/utils/extension/index.js @@ -224,11 +224,10 @@ body { } }` -// add html to the body of the current page +// Add HTML to the body of the current page var div = document.createElement('div'); div.innerHTML = ` Yuna AI -

Yuna

@@ -254,51 +253,54 @@ var submitButton = document.getElementById('submitButton'); var chatHistory = JSON.parse(localStorage.getItem('chatHistory')) || []; var closeBtn = document.getElementById('closeButton'); -function updateChatHistory(message, sender) { - chatHistory.push({ - sender: sender, - message: message - }); - localStorage.setItem('chatHistory', JSON.stringify(chatHistory)); - displayChatHistory(); -} - function displayChatHistory() { var chatArea = document.getElementById('chatArea'); chatArea.innerHTML = ''; // Clear previous chat bubbles chatHistory.forEach(function (entry) { var bubble = document.createElement('div'); - bubble.className = 'chat-bubble ' + (entry.sender === 'Yuki' ? 'you' : 'yuna'); + bubble.className = 'chat-bubble ' + (entry.sender === aiName ? 'you' : aiName2); bubble.textContent = entry.message; chatArea.appendChild(bubble); }); chatArea.scrollTop = chatArea.scrollHeight; // Scroll to the bottom } -function startYunaChat() { - var isDisplayed = popover.style.display === 'block'; - popover.style.display = isDisplayed ? 'none' : 'block'; - popover.style.opacity = isDisplayed ? 0 : 1; +function updateChatHistory(message, sender) { + chatHistory.push({ sender: sender, message: message }); + localStorage.setItem('chatHistory', JSON.stringify(chatHistory)); displayChatHistory(); -}; +} + +var aiName = "Yuki"; +var aiName2 = "Yuna"; submitButton.addEventListener('click', function () { var input = inputFieldYuna.value.trim(); if (input) { - updateChatHistory(input, 'Yuki'); - inputFieldYuna.value = ''; + updateChatHistory(input, aiName); + inputFieldYuna.value = ''; // Send message to API and get response + sendMessageToAPI(input).then(response => { + updateChatHistory(response, aiName2); + }).catch(error => { + console.error('Error:', error); + updateChatHistory('Error: Unable to get response from Yuna.', aiName2); + }); } - - // Simulate Yuna's response - setTimeout(function () { - updateChatHistory('I am a simple AI and cannot respond to that.', 'Yuna'); - }, 1000); }); inputFieldYuna.addEventListener('keypress', function (e) { if (e.key === 'Enter') { - updateChatHistory(inputFieldYuna.value, 'Yuki'); - inputFieldYuna.value = ''; + var input = inputFieldYuna.value.trim(); + if (input) { + updateChatHistory(input, aiName); + inputFieldYuna.value = ''; // Send message to API and get response + sendMessageToAPI(input).then(response => { + updateChatHistory(response, aiName2); + }).catch(error => { + console.error('Error:', error); + updateChatHistory('Error: Unable to get response from Yuna.', aiName2); + }); + } } }); @@ -310,12 +312,20 @@ clearChatButton.addEventListener('click', function () { localStorage.removeItem('chatHistory'); chatHistory = []; inputFieldYuna.value = ''; + displayChatHistory(); }); popover.querySelector('.popover-body').appendChild(clearChatButton); // Initialize chat history display on load window.onload = displayChatHistory; +function startYunaChat() { + var isDisplayed = popover.style.display === 'block'; + popover.style.display = isDisplayed ? 'none' : 'block'; + popover.style.opacity = isDisplayed ? 0 : 1; + displayChatHistory(); +} + floatingImageYuna.addEventListener('click', startYunaChat); closeBtn.addEventListener('click', startYunaChat); @@ -332,7 +342,6 @@ if (document.querySelector('#heart-yuna')) { var styleHeart = document.createElement('style'); styleHeart.appendChild(document.createTextNode(cssHeart)); document.head.appendChild(styleHeart); - document.querySelector('#search-input-yuna').addEventListener('keypress', function (event) { if (event.key === 'Enter') { event.preventDefault(); @@ -340,10 +349,132 @@ if (document.querySelector('#heart-yuna')) { window.open("https://www.google.com/search?q=" + encodeURIComponent(query), '_parent'); } }); - document.querySelector('#heart-yuna').addEventListener('click', function (event) { event.preventDefault(); const query = document.querySelector('#search-input-yuna').value; window.open("https://www.google.com/search?q=" + encodeURIComponent(query), '_parent'); }); } + +async function sendMessageToAPI(messageContent) { + const serverEndpoint = 'https://127.0.0.1:4848/message'; + const cookieMine = 'session=.eJwljkuKwzAQRK8ieh0GWZ9uy6eY_RBCSyrFBudD5FmF3H0EsyqKehTvTZe2a1_Rafl5kzlG0A296xV0ou8d2mH2x9Vsd3M8jJYyRnOsWzfPwXzR-XM-jZMX-kpL071j1K3SQhk5g12bohdMhTlxTkkQhefkHQSzxuxKjhysL16tDeprc1WsT8pVLSebYuVYJg2zxJAzS0FEyOKaAA7Wc0oeEm2dJ-Q0TxZF4LmW4X_57Xj922i9bXf6_AFRCkhZ.ZqmlHQ.g1zM1y6zu5JLrzIMMarqeORuqsY'; + const headers = { 'Content-Type': 'application/json', 'Cookie': cookieMine }; + + // Prepare the chat history and prompt template + const history = chatHistory.map(entry => `${entry.sender}: ${entry.message}`).join('\n'); + messageContent = `### System:\nYou're a helpful AI\n\n### Dialog:${history}\n${aiName}: ${messageContent}\n${aiName2}: `; + + const body = JSON.stringify({ + chat: 'history_template:general.json', + text: messageContent, + useHistory: false, + template: null, + speech: false, + yunaConfig: {}, + stream: false + }); + + try { + const response = await fetch(serverEndpoint, { + method: 'POST', + headers, + body, + credentials: 'include' + }); + + if (!response.ok) { + throw new Error(`HTTP error! status: ${response.status}`); + } + + const data = await response.json(); + console.log('Response:', data); + return data.response; // Assuming the API returns the response in a field named 'response' + } catch (error) { + console.error('Error:', error); + throw error; + } +} + +class ChatAPI { + constructor(messageManagerInstance) { + this.messageManager = messageManagerInstance; + this.createUI(); + this.chatHistory = "### System Message (constant):\nYou're a helpful AI\n\n"; + } + + createUI() { + const chatHTML = ` +
+
+ Chat with Yuna AI + +
+
+
+
+ + +
+
+
+ `; + + document.body.insertAdjacentHTML('beforeend', chatHTML); + + // Add event listeners + document.getElementById('close-chat').addEventListener('click', () => this.toggleChat()); + document.getElementById('send-message').addEventListener('click', () => this.sendMessage()); + document.getElementById('user-input').addEventListener('keypress', (e) => { + if (e.key === 'Enter') this.sendMessage(); + }); + + // Add Bootstrap CSS + const link = document.createElement('link'); + link.rel = 'stylesheet'; + link.href = 'https://cdn.jsdelivr.net/npm/bootstrap@5.3.0/dist/css/bootstrap.min.css'; + document.head.appendChild(link); + } + + toggleChat() { + const chat = document.getElementById('floating-chat'); + chat.style.display = chat.style.display === 'none' ? 'block' : 'none'; + } + + async sendMessage() { + const input = document.getElementById('user-input'); + const message = input.value.trim(); + if (message) { + this.addMessage('You', message); + input.value = ''; + + this.chatHistory += `### Question:\n${message}\n\n`; + + const response = await sendMessageToAPI(message); + + this.chatHistory += `### Answer:\n${response}\n\n`; + this.addMessage('Yuna', response); + } + } + + addMessage(sender, message) { + const chatMessages = document.getElementById('chat-messages'); + const messageElement = document.createElement('div'); + messageElement.className = `mb-2 ${sender === 'You' ? 'text-end' : ''}`; + messageElement.innerHTML = ` + + ${sender} + +
${message}
+ `; + chatMessages.appendChild(messageElement); + chatMessages.scrollTop = chatMessages.scrollHeight; + } + + getChatHistory() { + return this.chatHistory; + } +} + +// Initialize the ChatAPI +const chatAPI = new ChatAPI(); \ No newline at end of file diff --git a/app/chrome/manifest.json b/lib/utils/extension/manifest.json similarity index 100% rename from app/chrome/manifest.json rename to lib/utils/extension/manifest.json diff --git a/app/chrome/yuna-ai-128.png b/lib/utils/extension/yuna-ai-128.png similarity index 100% rename from app/chrome/yuna-ai-128.png rename to lib/utils/extension/yuna-ai-128.png diff --git a/lib/utils/yuna-tts b/lib/utils/yuna-tts new file mode 100755 index 0000000..39513c1 Binary files /dev/null and b/lib/utils/yuna-tts differ diff --git a/lib/utils/yuna-tts.m b/lib/utils/yuna-tts.m new file mode 100644 index 0000000..ce96ac6 --- /dev/null +++ b/lib/utils/yuna-tts.m @@ -0,0 +1,17 @@ +#import + +int main() { + __block BOOL done = NO; + [AVSpeechSynthesizer requestPersonalVoiceAuthorizationWithCompletionHandler:^(AVSpeechSynthesisPersonalVoiceAuthorizationStatus status){ + // authorization popup should be visible now + done = YES; + }]; + + // Run the loop for a maximum of 10 seconds + NSDate *loopUntil = [NSDate dateWithTimeIntervalSinceNow:10.0]; + while (!done && [loopUntil timeIntervalSinceNow] > 0) { + [[NSRunLoop currentRunLoop] runMode:NSDefaultRunLoopMode beforeDate:[NSDate dateWithTimeIntervalSinceNow:0.1]]; + } + + return 0; +} \ No newline at end of file diff --git a/lib/vision.py b/lib/vision.py index 47ad715..7e355ce 100644 --- a/lib/vision.py +++ b/lib/vision.py @@ -1,33 +1,29 @@ -import json import os -from diffusers import StableDiffusionPipeline -from datetime import datetime from llama_cpp import Llama from llama_cpp.llama_chat_format import MoondreamChatHandler +from lib.audio import speak_text +from lib.generate import get_config -if os.path.exists("static/config.json"): - with open("static/config.json", 'r') as file: - config = json.load(file) +config = get_config() -yuna_model_dir = config["server"]["yuna_model_dir"] -agi_model_dir = config["server"]["agi_model_dir"] -model_id = f"{yuna_model_dir}yuna-ai-miru-v0.gguf" -model_id_eyes = f"{yuna_model_dir}yuna-ai-miru-eye-v0.gguf" - -if config["ai"]["vision"] == True: - chat_handler = MoondreamChatHandler(clip_model_path=model_id_eyes) - llm = Llama( - model_path=model_id, - chat_handler=chat_handler, - n_ctx=4096, - seed=-1, - n_batch=512, - n_gpu_layers=-1, - verbose=False, - ) +llm = Llama( + model_path="lib/models/agi/miru/" + config["server"]["miru_default_model"], + chat_handler=MoondreamChatHandler(clip_model_path="lib/models/agi/miru/" + config["server"]["eyes_default_model"]), + n_ctx=4096, + seed=config["ai"]["seed"], + n_batch=config["ai"]["batch_size"], + n_gpu_layers=config["ai"]["gpu_layers"], + n_threads=config["ai"]["threads"], + use_mlock=config["ai"]["use_mlock"], + flash_attn=config["ai"]["flash_attn"], + verbose=False, +) if config["ai"]["miru"] == True else "" if config["ai"]["art"] == True: - art = StableDiffusionPipeline.from_single_file(f'{agi_model_dir}art/{config["server"]["art_default_model"]}', safety_checker=None, load_safety_checker=None, guidance_scale=7.5, noise_scale=0.05, device=config["server"]["device"]) + from diffusers import StableDiffusionPipeline + from datetime import datetime + + art = StableDiffusionPipeline.from_single_file(f'lib/models/agi/art/{config["server"]["art_default_model"]}', safety_checker=None, load_safety_checker=None, guidance_scale=7.5, noise_scale=0.05, device=config["server"]["device"]) art.to(config["server"]["device"]) def create_image(prompt): @@ -40,12 +36,11 @@ def create_image(prompt): # Get the current time in milliseconds current_time_milliseconds = int((datetime.utcnow() - datetime(1970, 1, 1)).total_seconds() * 1000) image_name = str(current_time_milliseconds) + '-art' + '.png' - image.save(f"static/img/art/{image_name}") return image_name -def capture_image(image_path=None, prompt=None, use_cpu=False): +def capture_image(image_path=None, prompt=None, use_cpu=False, speech=False): # print the parameters print(f"image_path: {image_path}") print(f"prompt: {prompt}") @@ -67,4 +62,6 @@ def capture_image(image_path=None, prompt=None, use_cpu=False): ) answer = result['choices'][0]['message']['content'] + if speech: + speak_text(answer) return [answer, image_path] \ No newline at end of file diff --git a/login.html b/login.html index 802e237..d2fd32f 100644 --- a/login.html +++ b/login.html @@ -160,8 +160,7 @@

Delete Account

- - + \ No newline at end of file diff --git a/services.html b/services.html index d5c401e..b60d828 100644 --- a/services.html +++ b/services.html @@ -85,7 +85,7 @@ Login @@ -387,8 +387,7 @@ - - + \ No newline at end of file diff --git a/static/config.json b/static/config.json index 4f97871..cb9190f 100644 --- a/static/config.json +++ b/static/config.json @@ -4,16 +4,19 @@ "Yuki", "Yuna" ], + "hinmitsu": false, + "agi": false, "emotions": false, "art": false, - "vision": false, - "max_new_tokens": 128, + "miru": false, + "search": false, + "max_new_tokens": 512, "context_length": 2048, "temperature": 0.7, - "repetition_penalty": 1.11, + "repetition_penalty": 1.1, "last_n_tokens": 128, "seed": -1, - "top_k": 60, + "top_k": 100, "top_p": 0.92, "stop": [ "Yuki:", @@ -36,19 +39,23 @@ "server": { "port": "", "url": "", - "history": "db/history/", - "default_history_file": "history_template.json", + "default_history_file": "history_template:general.json", "images": "images/", - "yuna_model_dir": "lib/models/yuna/", - "yuna_default_model": "yuna-ai-v3-q6_k.gguf", - "agi_model_dir": "lib/models/agi/", + "yuna_default_model": "yuna-ai-v3-q5_k_m.gguf", + "miru_default_model": "yuna-ai-miru-v0.gguf", + "eyes_default_model": "yuna-ai-miru-eye-v0.gguf", + "voice_default_model": "yuna-ai-voice-v1", "art_default_model": "yuna_ai_anime.safetensors", "device": "mps", "yuna_text_mode": "native", - "yuna_audio_mode": "fast" + "yuna_audio_mode": "siri", + "yuna_audio_name": "1.wav", + "yuna_reference_audio": "audio.mp3", + "output_audio_format": "audio.aiff" }, "security": { "secret_key": "YourSecretKeyHere123!", - "encryption_key": "zWZnu-lxHCTgY_EqlH4raJjxNJIgPlvXFbdk45bca_I=" + "encryption_key": "zWZnu-lxHCTgY_EqlH4raJjxNJIgPlvXFbdk45bca_I=", + "11labs_key": "Your11LabsKeyHere123!" } } diff --git a/static/css/about.css b/static/css/about.css deleted file mode 100644 index f387e63..0000000 --- a/static/css/about.css +++ /dev/null @@ -1,218 +0,0 @@ -* { - font-family: "kawai-font" !important; - -webkit-touch-callout: none; - -webkit-user-select: none; - -khtml-user-select: none; - -moz-user-select: none; - -ms-user-select: none; - user-select: none; -} - -body { - margin: 0; - background-image: url('/static/img/space.webp'); - background-size: cover; - background-position: center; - background-repeat: no-repeat; - background-attachment: fixed; - height: 100vh; - width: 100vw; - /* Center the content */ - display: flex; - justify-content: center; - align-items: center; - font-family: 'Arial', sans-serif; -} - -.modal-body { - color: black; -} - -p { - margin-bottom: 0; -} - -#waifu-card-container { - display: flex; - justify-content: center; - align-items: center; -} - -.waifu-card { - background: linear-gradient(135deg, #ffffff 0%, #ffffff 100%); - padding: 20px; - border-radius: 15px; - box-shadow: 0 4px 8px rgba(0, 0, 0, 0.1); - text-align: center; - width: 500px; - height: auto; - max-height: 90vh; - position: relative; - overflow: hidden; - margin: auto; - background-size: cover; - background-position: center; -} - -.waifu-card::before { - content: ''; - position: absolute; - top: 0; - right: 0; - bottom: 0; - left: 0; - background-image: url('/static/img/holobg.webp'); - background-size: cover; - background-repeat: no-repeat; - opacity: 0.9; - border-radius: 15px; -} - -.waifu-card img { - max-width: 100%; - max-height: 60vh; - border-radius: 10px; - position: relative; - z-index: 2; -} - -.waifu-info { - position: relative; - z-index: 2; - margin-top: 20px; -} - -.waifu-title { - color: #333; - font-size: 24px; - font-weight: bold; -} - -.waifu-description { - color: #666; - font-size: 16px; - margin-top: 10px; -} - -.cute-element { - color: #ff69b4; - font-size: 30px; - margin: 5px 0; -} - -.detail-panel { - background: rgba(255, 255, 255, 0.9); - padding: 10px; - border-radius: 10px; - position: absolute; - top: 20px; - right: 20px; - width: auto; - box-shadow: 0 2px 4px rgba(0, 0, 0, 0.1); -} - -.detail-text { - font-size: 12px; - color: #333; - line-height: 1.4; - text-align: left; -} - -/* Floating info blocks */ -.info-blocks { - position: absolute; - top: 20px; - left: 20px; - text-align: left; - color: #333; - z-index: 2; - display: flex; - flex-direction: column; -} - -.info-block { - display: flex; - justify-content: space-between; - align-items: center; - margin: 5px 0; - font-size: 14px; - background: rgba(255, 255, 255, 0.9); - padding: 5px 10px; - border-radius: 10px; -} - -.info-block p:first-child { - font-weight: bold; - margin-right: 10px; -} - -/* Waifu Title Block */ -.waifu-title-block { - display: flex; - align-items: center; - justify-content: center; -} - -/* Adjustments for image and text spacing */ -.waifu-card img { - max-width: 100%; - max-height: 70vh; - border-radius: 10px; - position: relative; - z-index: 1; -} - -.waifu-info { - position: relative; - z-index: 1; - margin-top: 10px; - color: white; -} - -.waifu-description { - color: white; - font-size: 24px; - font-weight: lighter; -} - -/* Ensure the card is not too tall */ -.waifu-card { - max-height: 85vh; - display: flex; - flex-direction: column; - justify-content: space-between; -} - -.modal-content { - background-color: #fff0f5; /* Light pink background */ - border-radius: 15px; /* Rounded corners for the modal */ -} - -.modal-title { - color: #ff69b4; /* Cute pink color for the title */ -} - -.btn-primary { - background-color: #ff69b4; /* Match the button color to the title */ - border-color: #ff69b4; -} - -.btn-primary:hover { - background-color: #ff5f9d; /* A slightly darker pink on hover */ - border-color: #ff5f9d; -} - -/* Responsive design for smaller screens */ -@media (max-width: 768px) { - .waifu-card { - width: 90%; - margin: 20px auto; - } -} - -@font-face { - font-family: "kawai-font"; - src: url("/static/fonts/kawai-font.woff") format("woff"); - font-style: normal; - font-weight: normal; -} \ No newline at end of file diff --git a/static/css/index.css b/static/css/index.css index e3c155f..4e00e16 100644 --- a/static/css/index.css +++ b/static/css/index.css @@ -248,11 +248,6 @@ input:checked~.slider .off-label { justify-content: center; } -.title { - color: #ffffff; - margin-bottom: 20px; -} - .scroll-to-top { bottom: 9%; } @@ -375,12 +370,6 @@ body { background-color: #4b6cff; } -.user-card img { - width: 50px; - height: 50px; - border-radius: 50%; -} - .toggle-theme { margin-top: 30px; } @@ -438,7 +427,6 @@ body { background-color: #f4f4f4; padding: 10px 20px; color: #212121; - font-family: ui-sans-serif, system-ui, -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, "Helvetica Neue", Arial, "Noto Sans", sans-serif; font-size: 16px; line-height: 24px; display: flex; diff --git a/static/css/kawai-v11-2.css b/static/css/kawai-v11-2.css index 5d7ae68..07efa67 100644 --- a/static/css/kawai-v11-2.css +++ b/static/css/kawai-v11-2.css @@ -689,6 +689,8 @@ label { white-space: -pre-wrap; /* Opera 4-6 */ white-space: -o-pre-wrap; /* Opera 7 */ word-wrap: break-word; /* Internet Explorer 5.5+ */ + display: flex; + flex-direction: column; } #docs h1, diff --git a/static/img/hardscenebreak.png b/static/img/hardscenebreak.png deleted file mode 100644 index 7d00de5..0000000 Binary files a/static/img/hardscenebreak.png and /dev/null differ diff --git a/static/img/holobg.webp b/static/img/holobg.webp deleted file mode 100644 index 079fdaa..0000000 Binary files a/static/img/holobg.webp and /dev/null differ diff --git a/static/img/space.webp b/static/img/space.webp deleted file mode 100644 index d7ec906..0000000 Binary files a/static/img/space.webp and /dev/null differ diff --git a/static/js/bootstrap/bootstrap.min.js b/static/js/bootstrap.min.js similarity index 100% rename from static/js/bootstrap/bootstrap.min.js rename to static/js/bootstrap.min.js diff --git a/static/js/bootstrap/script.min.js b/static/js/bootstrap/script.min.js deleted file mode 100644 index 731a1f7..0000000 --- a/static/js/bootstrap/script.min.js +++ /dev/null @@ -1 +0,0 @@ -!function(){"use strict";var e=document.querySelector(".scroll-to-top");e&&window.addEventListener("scroll",(function(){var o=window.pageYOffset;e.style.display=o>100?"block":"none"}));var o=document.querySelector("#mainNav");if(o){var n=o.querySelector(".navbar-collapse");if(n){var t=new bootstrap.Collapse(n,{toggle:!1}),r=n.querySelectorAll("a");for(var a of r)a.addEventListener("click",(function(e){t.hide()}))}var c=function(){(void 0!==window.pageYOffset?window.pageYOffset:(document.documentElement||document.body.parentNode||document.body).scrollTop)>100?o.classList.add("navbar-shrink"):o.classList.remove("navbar-shrink")};c(),document.addEventListener("scroll",c)}}(); \ No newline at end of file diff --git a/static/js/creator.js b/static/js/creator.js index 8d015fd..4227f1c 100644 --- a/static/js/creator.js +++ b/static/js/creator.js @@ -1,16 +1,16 @@ // Get the necessary elements -const promptTemplateTextarea = document.querySelector('#freeform-prompt-template'); -const bodyTextTextarea = document.querySelector('.body-text'); -const resultTextarea = document.querySelector('.result-container'); -const submitButton = document.getElementById('send-create-freeform'); +const promptTemplateTextarea = document.querySelector('#article-prompt-template'); +const bodyTextTextarea = document.querySelector('#body-text-article-container'); +const resultTextarea = document.querySelector('#result-create-article'); +const submitButton = document.getElementById('send-create-article'); // Set the default text for the Prompt Template block const defaultPromptTemplate = promptTemplateManager.buildPrompt('himitsuAssistant'); -promptTemplateTextarea.value = defaultPromptTemplate; +promptTemplateTextarea.value = defaultPromptTemplate.replace('### Instruction:\n', '### Instruction:\n{body_text}'); // Function to send the request to the server async function sendRequest() { - activeElement = document.getElementById('body-text-freeform-container'); + activeElement = document.getElementById('body-text-article-container'); const bodyText = bodyTextTextarea.value; const promptTemplate = promptTemplateTextarea.value.replace('{body_text}', bodyText); @@ -18,7 +18,7 @@ async function sendRequest() { // Clear the result textarea before starting resultTextarea.value = ''; - messageManagerInstance.sendMessage(promptTemplate, false, imageData = '', url = '/message', naked = false, stream = true, outputElement = resultTextarea); + messageManagerInstance.sendMessage(promptTemplate, null, imageData = '', url = '/message', naked = false, stream = true, outputElement = resultTextarea); } // Add an event listener to the submit button @@ -29,7 +29,7 @@ const presentationPromptTemplateTextarea = document.getElementById('presentation const presentationUserInputTextarea = document.getElementById('presentation-user-input'); const presentationOutputTextarea = document.getElementById('presentation-output-text'); const presentationDraftTextarea = document.getElementById('presentation-draft-text'); -const generateButton = document.getElementById('send-create-freeform'); +const generateButton = document.getElementById('send-create-article'); const copyToDraftButton = document.getElementById('copy-to-draft'); // Set the default text for the Presentation Prompt Template block diff --git a/static/js/himitsu.js b/static/js/himitsu.js deleted file mode 100644 index 7d53c2f..0000000 --- a/static/js/himitsu.js +++ /dev/null @@ -1,196 +0,0 @@ -var name1; -var name2; - -async function loadConfig() { - const response = await fetch('/static/config.json'); - const data = await response.json(); - name1 = data.ai.names[0]; - name2 = data.ai.names[1]; -} - - -class PromptTemplate { - constructor(fields, templateInputs) { - this.fields = fields; - this.templateInputs = templateInputs; - } - - generateElements() { - const form = document.getElementById("Himitsu"); - form.innerHTML = ''; - - this.templateInputs.forEach(input => { - const label = document.createElement("label"); - label.setAttribute("for", input.id); - label.textContent = `${input.label}:`; - - const newElement = document.createElement(input.type === 'select' ? "select" : "input"); - newElement.setAttribute("id", input.id); - newElement.setAttribute("name", input.id); - - if (input.type === 'select') { - input.options.forEach(option => { - const optionElement = document.createElement("option"); - optionElement.value = option.toLowerCase().replace(/\s+/g, '_'); - optionElement.textContent = option; - newElement.appendChild(optionElement); - }); - } else if (input.type === 'text') { - newElement.setAttribute("type", "text"); - newElement.setAttribute("placeholder", input.placeholder || ''); - } - - form.appendChild(label); - form.appendChild(newElement); - }); - } -} - -const commonOptions = [ - { id: 'audience', options: ['General', 'Knowledgeable', 'Expert', 'Other'] }, - { id: 'intent', options: ['Inform', 'Describe', 'Convince', 'Tell A Story', 'Other'] }, - { id: 'formality', options: ['Informal', 'Neutral', 'Formal', 'Other'] }, - { id: 'tone', options: ['Neutral', 'Friendly', 'Confident', 'Urgent', 'Joyful', 'Analytical', 'Optimistic', 'Other'] }, - { id: 'type', options: ['Blog Post', 'Email', 'Essay', 'Article', 'Description', 'Social Media Post', 'Document', 'Tutorial', 'Review', 'Creative Writing', 'Presentation', 'Speech', 'Research', 'Other'] } -]; - -const textInput = [{ id: 'text', label: 'Text', type: 'input' }]; - -const writer = new PromptTemplate([...commonOptions, { id: 'domain', options: ['Academic', 'Business', 'General', 'Email', 'Casual', 'Creative', 'Other'] }], textInput); -const paraphrase = new PromptTemplate(commonOptions, textInput); -const decisionMaking = new PromptTemplate([{ id: 'Mood', options: ['Good', 'Bad', 'Neutral'] }], textInput); -const himitsu = new PromptTemplate([{ id: 'question_type', options: ['Curiosity', 'Confusion', 'Research', 'Other'] }], [{ id: 'text', label: 'Text', type: 'text' }]); -const dialog = new PromptTemplate([{ id: 'text', label: 'Question', type: 'input' }], textInput); -const search = new PromptTemplate([{ id: 'text', label: 'Question', type: 'input' }], textInput); - -let currentPrompt = 'dialog'; -let currentPromptName = 'dialog'; - -function changeTemplate() { - const templateSelect = document.getElementById("templateSelect"); - const selectedTemplate = templateSelect.options[templateSelect.selectedIndex].value; - - // Generate select elements based on the selected template - const templateMap = { - 'writer': { - prompt: writer, - name: 'writer' - }, - 'paraphrase': { - prompt: paraphrase, - name: 'paraphrase' - }, - 'decisionMaking': { - prompt: decisionMaking, - name: 'decisionMaking' - }, - 'himitsu': { - prompt: himitsu, - name: 'himitsu' - }, - "search": { - prompt: search, - name: 'search' - } - }; - - if (templateMap[selectedTemplate] == "dialog") { - // Generate select elements based on the selected template - } else if (templateMap[selectedTemplate]) { - currentPrompt = templateMap[selectedTemplate].prompt; - currentPromptName = templateMap[selectedTemplate].name; - } -} - -function generateText() { - const selectedValues = {}; - - if (isHimitsu.toString() == 'true') { - // Handle template inputs - himitsuCopilot.templateInputs.forEach((input) => { - const element = document.getElementById(input.id); - if (element) { - selectedValues[input.id] = element.value; - } - }) - - } else { - // Handle template inputs - currentPrompt.templateInputs.forEach((input) => { - const element = document.getElementById(input.id); - if (element) { - selectedValues[input.id] = element.value; - } - }) - } - - const generatedText = Object.entries(selectedValues) - .map(([key, value]) => `${key.charAt(0).toUpperCase() + key.slice(1)}: ${value}`) - .join('\n'); - - console.log(generatedText); - - // Send generatedText to the server - sendGeneratedTextToServer(generatedText); -} - -async function sendGeneratedTextToServer(generatedText) { - const templateSelect = document.getElementById("templateSelect"); - const selectedTemplate = templateSelect.value; - - removeHimitsu(generatedText); - - const sendRequest = async (text, template) => { - const response = await fetch(`${server_url + server_port}/message`, { - method: 'POST', - headers: { 'Content-Type': 'application/json' }, - body: JSON.stringify({ - chat: selectedFilename, - text: text, - template: template, - }), - }); - - return await response.json(); - }; - - try { - const data = await sendRequest(generatedText, isHimitsu ? "himitsuCopilotGen" : selectedTemplate); - - if (isHimitsu.toString() === 'true') { - const data2 = await sendRequest(data.response, selectedTemplate); - messageManagerInstance.createMessage(name2, data2.response); - } else { - messageManagerInstance.createMessage(name2, data.response); - } - - messageManagerInstance.removeBr(); - messageManagerInstance.removeTypingBubble(); - loadConfig(); - messageManagerInstance.addBr(); - playAudio(audioType = 'message'); - - if (isTTS.toString() === 'true') { - playAudio(); - } - } catch (error) { - messageManagerInstance.removeTypingBubble(); - loadConfig(); - messageManagerInstance.createMessage(name2, error); - playAudio(audioType = 'error'); - } -} - -function removeHimitsu(msg) { - // Select the form element with the ID 'Himitsu' - var formElement = document.getElementById('Himitsu'); - - // Get the parent 'pre' element of the form - var preElement = formElement.parentNode; - - // Clear the contents of the 'pre' element - preElement.innerHTML = ''; - - // Set the text content of the 'pre' element to 'Hello World' - preElement.innerHTML = msg; -} \ No newline at end of file diff --git a/static/js/index.js b/static/js/index.js index 4243418..e96b42a 100644 --- a/static/js/index.js +++ b/static/js/index.js @@ -13,16 +13,12 @@ var isYunaListening = false; let mediaRecorder; let audioChunks = []; var activeElement = null; -//kawaiAutoScale(); // Global variable to track the state of Streaming Chat Mode let isStreamingChatModeEnabled = false; // Function to handle the toggle switch change -document.getElementById('streamingChatMode').addEventListener('change', function() { - isStreamingChatModeEnabled = this.checked; - console.log('Streaming Chat Mode is ' + (isStreamingChatModeEnabled ? 'enabled' : 'disabled')); -}); +document.querySelector('#streamingChatMode').onchange = e => isStreamingChatModeEnabled = e.target.checked; const buttonAudioRec = document.querySelector('#buttonAudioRec'); const iconAudioRec = buttonAudioRec.querySelector('#iconAudioRec'); @@ -35,7 +31,7 @@ buttonAudioRec.addEventListener('click', () => { } }); -function startRecording() { +function startRecording(withImage = false, imageDataURL, imageName, messageForImage) { navigator.mediaDevices.getUserMedia({ audio: true }) .then(stream => { mediaRecorder = new MediaRecorder(stream); @@ -52,7 +48,12 @@ function startRecording() { mediaRecorder.addEventListener('stop', () => { const audioBlob = new Blob(audioChunks, { type: 'audio/wav' }); - sendAudioToServer(audioBlob); + + if (withImage) { + sendAudioToServer(audioBlob, true, imageDataURL, imageName, messageForImage); + } else { + sendAudioToServer(audioBlob); + } audioChunks = []; }); }) @@ -70,7 +71,7 @@ function stopRecording() { isRecording = false; } -function sendAudioToServer(audioBlob) { +function sendAudioToServer(audioBlob, withImage = false, imageDataURL, imageName, messageForImage) { const formData = new FormData(); formData.append('audio', audioBlob); formData.append('task', 'transcribe'); @@ -86,7 +87,11 @@ function sendAudioToServer(audioBlob) { return response.json(); }) .then(data => { - messageManagerInstance.sendMessage(data.text, kanojo.buildKanojo(),'', '/message', false, false, isStreamingChatModeEnabled); + if (withImage) { + askYunaImage = messageManagerInstance.sendMessage(data.text, kanojo.buildKanojo(), [imageDataURL, imageName, data.text], '/image', false, false, isStreamingChatModeEnabled); + } else { + messageManagerInstance.sendMessage(data.text, kanojo.buildKanojo(),'', '/message', false, false, isStreamingChatModeEnabled); + }; }) .catch(error => { console.error('Error sending audio to server', error); @@ -94,29 +99,23 @@ function sendAudioToServer(audioBlob) { } async function loadConfig() { - const { ai: { names: [name1, name2] } } = await (await fetch('/static/config.json')).json(); - document.getElementById('input_text').placeholder = `Ask ${name2}...`; -} - -function changeHimitsuState() { - isHimitsu = !isHimitsu; -} - -function downloadVariableAsFile(variableContent, filename) { - // Create a Blob with the variable content - const blob = new Blob([variableContent], { type: 'text/plain' }); - - // Create an anchor element and trigger a download - const anchor = document.createElement('a'); - const url = URL.createObjectURL(blob); - anchor.href = url; - anchor.download = filename; - document.body.appendChild(anchor); - anchor.click(); - - // Clean up by revoking the Object URL and removing the anchor element - document.body.removeChild(anchor); - URL.revokeObjectURL(url); + let config; + // Check if 'config' exists in localStorage + if (localStorage.getItem('config')) { + // Parse the 'config' from localStorage + config = JSON.parse(localStorage.getItem('config')); + } else { + // Fetch the config from '/static/config.json' and parse it + config = await (await fetch('/static/config.json')).json(); + // Store the fetched config in localStorage for future use + localStorage.setItem('config', JSON.stringify(config)); + } + // Extract names from the config + const { ai: { names: [first, second] } } = config; + name1 = first; + name2 = second; + // Set the placeholder using the second name + document.getElementById('input_text').placeholder = `Ask ${second}...`; } class messageManager { @@ -196,24 +195,17 @@ class messageManager { async sendMessage(message, template, imageData = '', url = '/message', naked = false, stream = isStreamingChatModeEnabled || false, outputElement = '') { this.inputText = document.getElementById('input_text'); const messageContent = message || this.inputText.value; - const userMessageElement = this.createMessage(name1, messageContent); - this.createTypingBubble(naked); + var userMessageElement; + if (template !== null) { + userMessageElement = this.createMessage(name1, messageContent); + this.createTypingBubble(naked); + } if (url === '/message') { let result = ''; const decoder = new TextDecoder(); const serverEndpoint = `${server_url + server_port}${url}`; const headers = { 'Content-Type': 'application/json' }; - console.log({ - chat: selectedFilename, - text: messageContent, - useHistory: kanojo.useHistory, - template: (typeof template !== 'undefined') ? template : (this.inputText.value ? kanojo.buildKanojo() : null), - speech: isYunaListening, - yunaConfig: config_data, - stream - }); - const body = JSON.stringify({ chat: selectedFilename, text: messageContent, @@ -223,7 +215,6 @@ class messageManager { yunaConfig: config_data, stream }); - this.inputText.value = ''; try { @@ -255,11 +246,13 @@ class messageManager { } this.removeTypingBubble(); - console.log('Final result:', result); } else { const data = await response.json(); - this.removeTypingBubble(); - this.createMessage(name2, data.response); + + if (template !== null) { + this.removeTypingBubble(); + this.createMessage(name2, data.response); + } } if (isYunaListening) { @@ -311,7 +304,7 @@ class messageManager { const [imageDataURL, imageName, messageForImage] = imageData; const serverEndpoint = `${server_url + server_port}/image`; const headers = { 'Content-Type': 'application/json' }; - const body = JSON.stringify({ image: imageDataURL, name: imageName, message: messageForImage, task: 'caption', chat: selectedFilename}); + const body = JSON.stringify({ image: imageDataURL, name: imageName, message: messageForImage, task: 'caption', chat: selectedFilename, speech: isYunaListening }); fetch(serverEndpoint, { method: 'POST', headers, body }) .then(response => response.ok ? response.json() : Promise.reject('Error sending captured image.')) @@ -323,6 +316,9 @@ class messageManager { const imageResponse = `${data.message}`; this.createMessage(name2, imageResponse); + // play audio + playAudio(); + return imageCaption; }) .catch(error => { @@ -358,7 +354,6 @@ function playAudio(audioType = 'tts') { } } -// Other functions (clearHistory, loadHistory, downloadHistory) go here if needed. function formatMessage(messageData) { const messageDiv = document.createElement('div'); loadConfig(); @@ -454,15 +449,13 @@ function setMessagePopoverListeners() { }); }); } - -// run the setMessagePopoverListeners function with a delay of 1 second -setTimeout(setMessagePopoverListeners, 500); +setTimeout(setMessagePopoverListeners, 200); class HistoryManager { constructor(serverUrl, serverPort, defaultHistoryFile) { this.serverUrl = serverUrl || 'https://localhost:'; this.serverPort = serverPort || 4848; - this.defaultHistoryFile = defaultHistoryFile || 'history_template.json'; + this.defaultHistoryFile = defaultHistoryFile || 'history_template:general.json'; this.messageContainer = document.getElementById('message-container'); } @@ -631,11 +624,6 @@ class HistoryManager { }); } - closePopup(popupId) { - const popup = document.getElementById(popupId); - popup?.remove(); - } - // Placeholder for loadSelectedHistory method loadSelectedHistory(filename) { const selectedFilename = filename || this.defaultHistoryFile; @@ -655,8 +643,6 @@ class HistoryManager { .catch(error => { console.error('Error loading selected history file:', error); }); - - this.closePopupsAll(); } } @@ -668,6 +654,8 @@ function initializeVideoStream() { var videoStream = null; // To hold the stream globally var facingMode = "user"; // Default facing mode + localVideo.style.transform = "scaleX(-1)"; + // Function to start or restart the video stream with the given facingMode function startVideo() { // First, stop any existing video stream @@ -728,11 +716,6 @@ function initializeVideoStream() { // Initialize the video stream functionality after a delay setTimeout(initializeVideoStream, 500); -function closePopup(popupId) { - var popup = document.getElementById(popupId); - popup.remove(); -} - function scrollMsg() { objDiv = document.getElementById("message-container"); objDiv.scrollTop = objDiv.scrollHeight; @@ -743,8 +726,6 @@ async function drawArt() { const imagePrompt = prompt('Enter a prompt for the image:'); loadConfig(); - closePopupsAll(); - messageContainer.insertAdjacentHTML('beforeend', formatMessage({ name: name1, message: imagePrompt })); messageContainer.insertAdjacentHTML('beforeend', typingBubble); scrollMsg(); @@ -769,63 +750,53 @@ async function drawArt() { } // Add an event listener to the "Capture Image" button -function captureImage() { +async function captureImage() { var localVideo = document.getElementById('localVideo'); var captureCanvas = document.getElementById('capture-canvas'); var captureContext = captureCanvas.getContext('2d'); - messageContainer = document.getElementById('message-container'); + var messageContainer = document.getElementById('message-container'); // Set the canvas dimensions to match the video element captureCanvas.width = localVideo.videoWidth; captureCanvas.height = localVideo.videoHeight; - // Draw the current frame from the video onto the canvas - captureContext.drawImage(localVideo, 0, 0, captureCanvas.width, captureCanvas.height); - - captureCanvas = document.getElementById('capture-canvas'); - imageDataURL = captureCanvas.toDataURL('image/png'); // Convert canvas to base64 data URL + // Apply mirroring transformation to the canvas context + captureContext.save(); // Save the current state + captureContext.scale(-1, 1); // Flip horizontally + captureContext.drawImage(localVideo, -captureCanvas.width, 0, captureCanvas.width, captureCanvas.height); // Draw the video frame + captureContext.restore(); // Restore the original state - messageForImage = prompt('Enter a message for the image:'); + var imageDataURL = captureCanvas.toDataURL('image/png'); // Convert canvas to base64 data URL - // generate a random image name using current timestamp + let messageForImage = ''; var imageName = new Date().getTime().toString(); - closePopupsAll(); - askYunaImage = messageManagerInstance.sendMessage('', kanojo.buildKanojo(), [imageDataURL, imageName, messageForImage], '/image', false, false, isStreamingChatModeEnabled); + if (isYunaListening) { + // Start recording + startRecording(true, imageDataURL, imageName, messageForImage); + + return true; + } else { + messageForImage = prompt('Enter a message for the image:'); + askYunaImage = messageManagerInstance.sendMessage(messageForImage, kanojo.buildKanojo(), [imageDataURL, imageName, messageForImage], '/image', false, false, isStreamingChatModeEnabled); + } } // Modify the captureImage function to handle file uploads async function captureImageViaFile(image=null, imagePrompt=null) { var imageUpload = ''; var file = ''; - if (image && !imagePrompt) { - imageUpload = document.getElementById('imageUpload'); - file = imageUpload.files[0]; - console.log('here'); - } else if (imagePrompt && image) { - file = image; - console.log('here2'); - } - - if (!file) { - alert('No file selected.'); - return; - } + file = image && !imagePrompt ? document.getElementById('imageUpload').files[0] : imagePrompt && image ? image : null; + if (!file) return alert('No file selected.'); const reader = new FileReader(); reader.onloadend = async function () { const imageDataURL = reader.result; var messageForImage = ''; - if (imagePrompt) { - messageForImage = imagePrompt; - } else { - messageForImage = prompt('Enter a message for the image:'); - } + messageForImage = prompt('Enter a message for the image:'); const imageName = Date.now().toString(); - - closePopupsAll(); messageManagerInstance.sendMessage('', kanojo.buildKanojo(), [imageDataURL, imageName, messageForImage], '/image', false, false, isStreamingChatModeEnabled); }; @@ -845,8 +816,6 @@ function captureAudioViaFile() { reader.onloadend = async function () { const audioDataURL = reader.result; const audioName = Date.now().toString(); - - closePopupsAll(); }; const formData = new FormData(); @@ -854,14 +823,38 @@ function captureAudioViaFile() { formData.append('audio', file); formData.append('task', 'transcribe'); + const userQuestion = prompt("What's your question?"); + fetch('/audio', { method: 'POST', body: formData }) .then(response => response.json()) .then(data => { - console.log('The text in video:', data.text); - }) + const makeRAG = confirm("Do you want to make a RAG?"); + if (!makeRAG) { + var ragMessage = `Context Audio: "${data.text}"\nQuestion: ${userQuestion}` + document.getElementById('input_text').value = ragMessage; + messageManagerInstance.sendMessage(ragMessage); + } else { + const textBlob = new Blob([data.text], { type: 'text/plain' }); + const questionFormData = new FormData(); + questionFormData.append('text', textBlob, 'content.txt'); + questionFormData.append('query', userQuestion); + + fetch('/analyze', { + method: 'POST', + body: questionFormData + }) + .then(response => response.json()) + .then(result => { + console.log(result); + }) + .catch(error => { + console.error('Error:', error); + }); + } + }); reader.readAsDataURL(file); } @@ -899,19 +892,15 @@ function captureVideoViaFile() { captureFrame(); }, { once: true }); } else { - closePopupsAll(); + console.log('not implemented'); } } captureFrame(); }); - console.log('Frames captured:', videoFrames); - console.log('first frame:', videoFrames[0]); - // capture first frame using captureImageViaFile(image) function captureRessult = captureImageViaFile(videoFrames[0], 'Describe the image'); - console.log('Capture result:', captureRessult); const formData = new FormData(); formData.append('audio', file); @@ -940,8 +929,6 @@ function captureTextViaFile() { reader.onloadend = async function () { const textDataURL = reader.result; const textName = Date.now().toString(); - - closePopupsAll(); } const formData = new FormData(); @@ -1131,8 +1118,6 @@ function populateHistorySelect() { console.error('An error occurred:', error); }); break; - default: - console.log(`Unknown action: ${action} for file: ${fileName}`); } }); }); @@ -1160,13 +1145,11 @@ function loadSelectedHistory(selectedFilename) { body: JSON.stringify({ task: 'load', chat: selectedFilename }) }) .then(response => { - console.log(); if (!response.ok) throw new Error('Error loading selected history file.'); return response.json(); }) .then(data => { messageManagerInstance.displayMessages(data); - closePopupsAll(); }) .catch(error => { console.error('Error:', error); @@ -1174,40 +1157,55 @@ function loadSelectedHistory(selectedFilename) { } function duplicateAndCategorizeChats() { - const chatItems = document.querySelectorAll('#chat-items .collection-item'); - const collectionItems = document.createElement('div'); - collectionItems.id = 'collectionItems'; - collectionItems.classList.add('list-group'); - - const generalChatsDiv = document.createElement('div'); - const otherChatsDiv = document.createElement('div'); - - chatItems.forEach(item => { - const clonedItem = item.cloneNode(true); - const collectionName = clonedItem.querySelector('.collection-name').textContent; - (collectionName.includes(':general:') ? generalChatsDiv : otherChatsDiv).appendChild(clonedItem); - }); + const chatList = document.querySelector('#chat-items'); + const collectionItems = document.querySelector('#collectionItems'); + + // Clear existing content in collectionItems + collectionItems.innerHTML = ''; - collectionItems.append(generalChatsDiv, otherChatsDiv); - document.getElementById('collectionItems').replaceWith(collectionItems); -} + // Create an object to store categorized chats + const categories = { + 'Uncategorized': [] + }; -function fixDialogData() { - populateHistorySelect().then(() => { - loadSelectedHistory(); + // Check if chatList exists and has items + const items = chatList && chatList.children.length > 0 + ? chatList.querySelectorAll('.collection-item') + : document.querySelectorAll('.collection-item'); + + items.forEach(item => { + const chatName = item.querySelector('.collection-name').textContent; + const colonIndex = chatName.indexOf(':'); + + if (colonIndex !== -1) { + const folder = chatName.substring(colonIndex + 1).split('.')[0]; // Get the part after ':' and before '.' + if (!categories[folder]) { + categories[folder] = []; + } + categories[folder].push(item.cloneNode(true)); + } else { + categories['Uncategorized'].push(item.cloneNode(true)); + } }); - closePopupsAll(); -} + // Create category blocks and add chats + for (const [category, chats] of Object.entries(categories)) { + if (chats.length === 0) continue; // Skip empty categories -function closePopupsAll() { - var popups = document.querySelectorAll('.block-popup'); - popups.forEach(popup => { - popup.style.display = 'none'; - }); + const categoryBlock = document.createElement('div'); + categoryBlock.className = 'category-block mb-3'; + categoryBlock.innerHTML = `
${category}
`; + + const categoryList = document.createElement('ul'); + categoryList.className = 'list-group'; - var parameterContainer = document.getElementById('parameter-container'); - parameterContainer.innerHTML = ''; + chats.forEach(chat => { + categoryList.appendChild(chat); + }); + + categoryBlock.appendChild(categoryList); + collectionItems.appendChild(categoryBlock); + } } var myDefaultAllowList = bootstrap.Tooltip.Default.allowList; @@ -1238,35 +1236,45 @@ function handleTextFileClick() { document.getElementById('textUpload').click(); } -document.addEventListener('DOMContentLoaded', (event) => { - const switchInput = document.getElementById('customSwitch'); - const toastElement = document.getElementById('toggleToast'); - const toast = new bootstrap.Toast(toastElement); +document.addEventListener('DOMContentLoaded', loadConfig); - switchInput.addEventListener('change', () => { - toast.show(); - }); -}); +class NotificationManager { + constructor() { + this.dropdownMenu = document.querySelector('.dropdown-menu.dropdown-menu-end.dropdown-list.animated--grow-in'); + this.messages = []; + } -async function checkMe() { - const response = await fetch('/flash-messages'); - return response.json(); -} + add(message) { + this.messages.push(message); + this.render(); + } -document.addEventListener('DOMContentLoaded', loadConfig); + delete(message) { + this.messages = this.messages.filter(msg => msg !== message); + this.render(); + } -function importFlash(messages) { - const dropdownMenu = document.querySelector('.dropdown-menu.dropdown-menu-end.dropdown-list.animated--grow-in'); - dropdownMenu.innerHTML = messages.map(message => ` - -
- ${message} -
- - `).join(''); + clearAll() { + this.messages = []; + this.render(); + } + + render() { + this.dropdownMenu.innerHTML = this.messages.map(message => ` + +
+ ${message} +
+ + `).join(''); + } } -checkMe().then(importFlash).catch(console.error); +// Create an instance of NotificationManager +const notificationManagerInstance = new NotificationManager(); + +// Call the add method +notificationManagerInstance.add("Hello! Welcome to the chat room!"); function updateMsgCount() { setTimeout(() => { @@ -1298,25 +1306,13 @@ function deleteMessageFromHistory(message) { } return response.json(); }) - .then(responseData => { - console.log(responseData); - }) + .then(responseData => { alert(responseData); location.reload(); }) .catch(error => { console.error('An error occurred:', error); }); } } -document.querySelectorAll('.creatorStudio-tabs').forEach(tab => { - tab.addEventListener('click', function () { - document.querySelectorAll('.tab-pane').forEach(pane => { - pane.classList.add('d-none'); - }); - const target = document.querySelector(this.getAttribute('data-bs-target')); - target.classList.remove('d-none'); - }); -}); - // Function to adjust textarea height function adjustTextareaHeight(textarea) { textarea.style.height = 'auto'; @@ -1347,4 +1343,9 @@ function initializeTextareas() { document.addEventListener('DOMContentLoaded', initializeTextareas); // Also run it immediately in case the script is loaded after the DOM -initializeTextareas(); \ No newline at end of file +initializeTextareas(); + +function resetEverything() { + localStorage.clear(); + location.reload(); +} \ No newline at end of file diff --git a/static/js/search.js b/static/js/search.js index 266ac3c..ca6e729 100644 --- a/static/js/search.js +++ b/static/js/search.js @@ -22,9 +22,7 @@ document.getElementById('searchButton').addEventListener('click', function () { }) }) .then(response => response.json()) // Parse the JSON from the response - .then(data => { - console.log('Data:', data); - + .then(data => { // Extract the message and results from the data var message = data.message[0]; var results = data.message[1]; diff --git a/static/js/setup.js b/static/js/setup.js index f80c57b..0d3b79c 100644 --- a/static/js/setup.js +++ b/static/js/setup.js @@ -28,7 +28,7 @@ function createFormGroup(id, value) { return `
- +
`; } @@ -92,7 +92,9 @@ async function checkConfigData() { port: server_port } } = config_data); - fixDialogData(); + populateHistorySelect().then(() => { + loadSelectedHistory(); + }); }, 100); } else { try { @@ -111,7 +113,6 @@ async function checkConfigData() { await delay(100); openConfigParams(); } - closePopupsAll(); } checkConfigData(); @@ -123,68 +124,74 @@ document.addEventListener("keydown", function (event) { if (event.key === "Tab") { event.preventDefault(); toggleSidebar(); - kawaiAutoScale() + kawaiAutoScale(); return; } var inputs = Array.from(document.getElementsByTagName('input')); var textareas = Array.from(document.getElementsByTagName('textarea')); - // Check if Shift key is pressed along with the key and is not in the input field - if (!inputs.includes(document.activeElement) && !textareas.includes(document.activeElement) && event.shiftKey) { + // Check if Command key is pressed along with the key and is not in the input field + if (!inputs.includes(document.activeElement) && !textareas.includes(document.activeElement) && event.metaKey) { + var navSidebar = document.getElementsByClassName('side-link'); switch (event.key) { - case "H": + case "1": event.preventDefault(); // Prevent any default action - var navSidebar = document.getElementsByClassName('side-link'); - for (let j = 0; j < navSidebar.length; j++) { navSidebar[j].classList.remove('active'); } navSidebar[0].classList.add('active'); - OpenTab('1') - + OpenTab('1'); break; - case "L": + case "2": event.preventDefault(); // Prevent any default action - var navSidebar = document.getElementsByClassName('side-link'); - for (let j = 0; j < navSidebar.length; j++) { navSidebar[j].classList.remove('active'); } navSidebar[1].classList.add('active'); - OpenTab('2') + OpenTab('2'); break; - case "E": + case "3": event.preventDefault(); // Prevent any default action - var navSidebar = document.getElementsByClassName('side-link'); - for (let j = 0; j < navSidebar.length; j++) { navSidebar[j].classList.remove('active'); } navSidebar[2].classList.add('active'); - OpenTab('3') + OpenTab('3'); break; - case "P": + case "4": event.preventDefault(); // Prevent any default action - var navSidebar = document.getElementsByClassName('side-link'); - for (let j = 0; j < navSidebar.length; j++) { navSidebar[j].classList.remove('active'); } navSidebar[3].classList.add('active'); - OpenTab('4') + OpenTab('4'); break; - case "S": + case "5": event.preventDefault(); // Prevent any default action - var navSidebar = document.getElementsByClassName('side-link'); - for (let j = 0; j < navSidebar.length; j++) { navSidebar[j].classList.remove('active'); } navSidebar[4].classList.add('active'); + OpenTab('5'); + break; + case "6": + event.preventDefault(); // Prevent any default action + for (let j = 0; j < navSidebar.length; j++) { + navSidebar[j].classList.remove('active'); + } + navSidebar[5].classList.add('active'); settingsView.show(); break; - case "C": + case "7": + event.preventDefault(); // Prevent any default action + for (let j = 0; j < navSidebar.length; j++) { + navSidebar[j].classList.remove('active'); + } + navSidebar[6].classList.add('active'); + window.open('https://www.patreon.com/YukiArimo', '_blank'); + break; + case "Y": event.preventDefault(); // Prevent any default action // check if callYuna is open if (document.getElementById('videoCallModal').classList.contains('show')) { @@ -198,15 +205,14 @@ document.addEventListener("keydown", function (event) { // Check if Enter key is pressed if (event.key === "Enter") { - // Prevent default action for Enter to avoid submitting the form - event.preventDefault(); // Check if the message input is focused if (document.activeElement === document.getElementById('input_text')) { // Send the message - messageManagerInstance.sendMessage('') + event.preventDefault(); + messageManagerInstance.sendMessage(''); } } -}) +}); var callYuna = { myModal: new bootstrap.Modal(document.getElementById('videoCallModal'), {}), @@ -230,29 +236,6 @@ const tabs = document.querySelectorAll('.tab'); // Get all content sections const sections = document.querySelectorAll('.section'); -// Function to remove active class from all tabs and hide all sections -function resetActiveTabsAndHideSections() { - tabs.forEach(tab => { - tab.classList.remove('active'); - }); - sections.forEach(section => { - section.style.display = 'none'; - }); -} - -// Function to initialize tabs functionality -function initTabs() { - tabs.forEach((tab, index) => { - tab.addEventListener('click', () => { - resetActiveTabsAndHideSections(); - // Add active class to the clicked tab - tab.classList.add('active'); - // Display the corresponding section - sections[index].style.display = 'block'; - }); - }); -} - if (window.matchMedia("(max-width: 428px)").matches) { document.getElementsByClassName('scroll-to-top')[0].style.display = 'none'; } diff --git a/static/sw.js b/static/sw.js index d28e445..c286119 100644 --- a/static/sw.js +++ b/static/sw.js @@ -8,8 +8,8 @@ toolbox.precache([ "/static/js/himitsu.js", "/static/js/index.js", "/static/js/kawai-v11-2.js", - "/static/js/bootstrap/bootstrap.min.js", - "/static/js/bootstrap/script.min.js", + "/static/js/bootstrap.min.js", + "/static/js/script.min.js", "/static/fonts/kawai-font.woff", "/static/img/yuna-ai.png", "/static/img/yuna-girl-head.webp", diff --git a/yuna.html b/yuna.html index 06d2fdc..cc80922 100644 --- a/yuna.html +++ b/yuna.html @@ -36,7 +36,6 @@ - @@ -146,7 +145,8 @@ Search - + Notifications @@ -174,12 +174,12 @@
- +
+
-
+
-
-
-
Prompt Template
- -
-
-
-
-
Body Text
- +
+
+
+
Work Area
+ +
-
-
- -
- -
-
-
-
Result
- -
+
+
@@ -569,7 +555,7 @@
User Input
+ class="btn btn-primary">Create
@@ -597,7 +583,37 @@
Draft Text
- +
+
+
+
Prompt Template
+ +
+
+
+
+
Body Text
+ +
+
+
+ +
+ +
+ +
+
+
+
Result
+ +
+
+
@@ -644,7 +660,7 @@ id="iconAudioRec"> -
@@ -699,7 +715,7 @@ aria-controls="v-pills-general" aria-selected="true">General + aria-controls="v-pills-system" aria-selected="false">Account @@ -715,15 +731,24 @@
General
- +
- + +
+
+ +
+
Data
+
+ +
-
@@ -736,8 +761,13 @@
Mode
- + +
+ +
+ Reset
@@ -749,122 +779,108 @@
Mode
-
-
Data
-
- - -
-
-
Account
- -
-
-
-
-
-
-

- Change Password

- -
- -
-
- -
-
- -
- -
-
+
+
+
+
+
+

+ Change Password

+ +
+ +
+
+ +
+
+ +
+ +
+
-
-
-
-

- New User

- -
- -
-
- -
- -
-
+
+
+
+

+ New User

+ +
+ +
+
+ +
+ +
+
-
-
-
-

- Login

- -
- -
-
- -
- -
-
+
+
+
+

+ Login

+ +
+ +
+
+ +
+ +
+
-
-
-
-

- Delete Account

- -
- -
-
- -
- -
-
+
+
+
+

+ Delete Account

+ +
+ +
+
+ +
+ +
-
@@ -924,8 +940,8 @@
File Input