-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* Create README.md * added contribution guidelines * documented code * Corrected diagram
- Loading branch information
1 parent
9d52f28
commit 07ae17a
Showing
13 changed files
with
193 additions
and
9 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,37 @@ | ||
# Contributing | ||
|
||
First of all, a huge thank you for looking to contribute to this repository! This project was concieved and built at [FOSS Hack 4.0](https://fossunited.org/fosshack/2024), a hackathon for building open-source software. As such, everything from the architecture of this server to the code has been *hacked up* in a couple of hours. That is to say, this is messed up lol. There's tons of stuff to improve. | ||
|
||
## Improvements | ||
These are some of the stuff that we'd like to improve upon | ||
|
||
### Reduce memory footprint | ||
Any changes to reduce the overall memory footprint of the application are welcome. We're sure there are lots of `String` variables instead of string literals waiting for you to get started with. 😂 Just for context, there are 36 `.clone()` calls throughout the codebase right now! That's 36 deep copy operations waiting to be removed. | ||
|
||
Any changes improving performance in any other way are appreciated as well. Eliminating JavaScript is welcome. More on that below. | ||
|
||
### Complete overhaul? | ||
Sure, why not! As said earlier, this was hacked up and we're not sure this is the best approach to build this server. Right now, it works and that's all we know! If you find a way to overhaul it and make it better, please do so and open a PR! | ||
|
||
> Note: If your PR changes any of the endpoint behaviors or names, please coordinate with us by [messaging me on telegram](https://t.me/vishalds/). | ||
### Eliminating JavaScript | ||
The screenshot script can maybe be replaced, but we couldn't find an effective solution for that right now. The most effective workaround for this would be to switch the previews altogether to use an `<iframe />` in the site. We did try this during the hackathon but couldn't quite get it to work. However, that is outside the scope of this repository. We'd still need screenshots for the previews, so any attempts to move this to rust are appreciated. | ||
|
||
As for the HTML string builder script, there currently seem to be no effective solutions to parse HTML reliably with rust alone. We briefly tried [scraper](https://docs.rs/scraper/latest/scraper/) & [select](https://docs.rs/select/latest/select/) but both of these ended up not having solutions to edit the HTML content. They only allow us to parse a string into different elements. So, we resorted to use JS and native DOM for this. Any attempts to move this to rust are appreciated as well. | ||
|
||
### Containerization | ||
Usually, servers like these are containerized on deployment. However, since we're not familiar with setting up containers frequently, we couldn't risk doing this at the hackathon. | ||
|
||
So, any attempts to containerize this are welcome. A good solution seems to be [docker](https://docker.com/). We're open to anything though. | ||
|
||
### Improve code quality | ||
If you can find places to refactor, improve code quality, or do any such thing, feel free to open a PR. | ||
|
||
## How to contribute? | ||
Simple, just open a PR. Let the checks run. If the build check is successful and your description conveys the intent, then we'll probably merge it. | ||
|
||
You can also open an issue if you spot one. | ||
|
||
## Merging right now... | ||
Right now, the evaluation for FOSS Hack is still ongoing. As such, we won't be merging any PR's until August 20, 2024. But feel free to open one, we'll merge once the evaluation finishes. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,124 @@ | ||
<img src="https://github.com/user-attachments/assets/bc7fe8b8-ca25-4d9d-8de3-24a717068d9a" alt="zitefy" width="250" align="right"> | ||
|
||
# `server` 🖥️ | ||
|
||
> This is the backend server of zitefy, which manages all the user data, templates, website previews and basically everything. Written in rust, it uses the actix web framework with some other crates. | ||
## Table Of Contents | ||
* [What's this?](#whats-this) | ||
* [Role in the stack](#role-in-the-stack) | ||
* [Architectural Overview](#architectural-overview) | ||
* [Routes](#routes) | ||
* [Security](#security) | ||
* [Setup Guide](#setup-guide) | ||
* [Prerequisites](#prerequisites) | ||
* [Installation](#installation) | ||
* [Build & run](#build--run) | ||
* [Contributing](#contributing) | ||
|
||
## What's this? | ||
In order for the core functionalities to work, there should be a central system where templates, users & their sites along with other stuff like assets, seo data can come together. This server is the means for doing so. | ||
|
||
### Role in the stack | ||
* Maintain and store user data, site data & template data | ||
* Provide a storage mechanism for storing site assets, seo data, etc | ||
* Generate previews for templates, sites & the web based editor | ||
|
||
### Architectural Overview | ||
This is written from the high level architecture shown below. You can find some pointers below highlighting the core idea in a nutshell. Feel free to open a new issue if you find any part of this documentation confusing or hard to understand. | ||
|
||
![arch](https://github.com/user-attachments/assets/344ba61f-b1d4-48b2-8066-405e8716aac2) | ||
|
||
*In a nutshell...* | ||
* Each user can register themselves on the API. Once registered or logged in, the API will generate an access token that can then be used to access the secure endpoints. Registered users can create sites from the available templates. | ||
* In zitefy, only user data is kept private. This is an open platform and so, all source code & data related to all sites & templates will be exposed publicly. This is a reminder to not upload any data to a site that you don't want the world to see. | ||
* Templates are uploaded and managed from the [templates]() repository. The CI/CD uploads the templates to a dedicated directory on the server and a background task that runs once every hour will update the template data in the database. | ||
* The preview engine is basically two JS scripts. One is the [`scripts/builder.js`]() that builds an HTML string from the given data. Another is the [`scripts/screenshot.js`]() that takes a screenshot of the generated HTML. | ||
|
||
The intention was to keep this a pure rust codebase, but it turns out that there are no effective html parsers in rust. Neither are there any methods to take a screenshot of a webpage. So, we ended up writing two JS files for those and only those. The rest is pure rust. | ||
|
||
The server then binds with two ports. Then, [nginx](https://nginx.org/en/0) redirects traffic from both [api.zitefy.com](https://api.zitefy.com/docs/#/) & [zitefy.com](https://zitefy.com/) to the respective ports. There is a [systemd](https://systemd.io/) service that manages everything. | ||
|
||
It would be cool to containerize this server as it'd speed up CI/CD for future versions. But right now, the setup overhead seemed a bit too much for the hackathon so we skipped it. | ||
|
||
## Routes | ||
Routes are of 4 types: `site`, `template`, `user` & `proxy`. Yeah, unfortunately proxy is a category. Anthropic, regardless of maintaining a typescript SDK, [doesn't allow CORS](https://github.com/anthropics/anthropic-sdk-typescript/issues/410) in their API rendering it unusable for web apps. If this had worked, it would have reduced our code complexity as well, but since it doesn't, proxy is a thing. | ||
|
||
As it should be obvious, listing all the routes here seems tedious. So, they're documented in this [swagger UI](https://api.zitefy.com/docs/#/). | ||
|
||
## Security | ||
The focus up until now has been somehow getting this to work properly for the hackathon. Security wasn't the first thing in our mind when building this. As such, it is advised not to upload any sensitive info to zitefy. | ||
|
||
Here are the measures taken for security: | ||
* All passwords are encrypted using a hashing algorithm with a secret key. Without it, the API will not allow access to user data | ||
* All logins are regulated by access tokens which expire every 30 days. After 30 days, you'd have to login again to continue using the site | ||
* The secret key used for hashing, along with some other stuff, are stored in a separate file not exposed in this repository. | ||
|
||
If you find a security vulnerability, please do open an issue/PR and we'd be happy to accept it! | ||
|
||
## Setup Guide | ||
You can follow this guide to run the server on a machine of your choice. If you want to deploy, it's advisable to do so in a linux system. There is nothing OS specific about the code itself, and so it can build and run on any platform. However, it'd be better to deploy on a linux machine since the secrets and service file are setup to be managed by systemd. When you open a PR, it'll be much easier for us to evaluate. If you're on Windows, WSL would be more than enough. | ||
|
||
### Prerequisites | ||
* The **Rust toolchain**, including cargo. | ||
|
||
Install for linux/WSL by running this command | ||
``` | ||
$ curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh | ||
``` | ||
Alternatively, follow the instruction method for your platform [here](https://www.rust-lang.org/tools/install) | ||
|
||
* **Git CLI** | ||
|
||
For linux, skip this step. Install by running the executable for your platform available [here](https://git-scm.com/downloads) | ||
|
||
* **MongoDB Community Server** | ||
|
||
Install for Ubuntu by following [this guide](https://www.mongodb.com/docs/manual/tutorial/install-mongodb-on-ubuntu/). For other operating systems, follow the corresponding tutorial from the [tutorials](https://www.mongodb.com/docs/manual/installation/#mongodb-installation-tutorials) page. If you want to visualize the data better, install MongoDB Compass by following [this guide](https://www.mongodb.com/docs/compass/current/install/). | ||
|
||
* **bun** | ||
|
||
Install for linux/WSL or macOS by running this command | ||
``` | ||
$ curl -fsSL https://bun.sh/install | bash | ||
``` | ||
Alternatively, you can follow the instructions for your platform [here](https://bun.sh/). | ||
|
||
### Installation | ||
1. Clone the repository to your machine. | ||
|
||
``` | ||
git clone https://github.com/zitefy/server.git | ||
``` | ||
|
||
2. Install JavaScript dependencies | ||
|
||
``` | ||
bun install puppeteer | ||
``` | ||
``` | ||
bun run puppeteer browsers install chrome | ||
``` | ||
3. Copy the chrome executable path to line 8 in [`scripts/screenshot.js`](). | ||
|
||
### Build & run | ||
1. Create an env file with all your secrets. An example file would look like this | ||
|
||
```env | ||
MONGODB_URI="mongodb://127.0.0.1:27017/" | ||
API_ADDR=127.0.0.1:7878 | ||
SERVER_ADDR=127.0.0.1:7979 | ||
SECRET_KEY="xxxx" | ||
ANTHROPIC_KEY='xxxx' | ||
``` | ||
You can get your anthropic API key [here](https://console.anthropic.com/settings/keys) | ||
2. Build and run the server | ||
|
||
``` | ||
cargo run | ||
``` | ||
|
||
That's about it, the server should be up and running locally in your machine. | ||
|
||
## Contributing | ||
There's tons of stuff to contribute. Please refer to the [contributing guide](https://github.com/zitefy/server/blob/main/CONTRIBUTING.md). |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file was deleted.
Oops, something went wrong.