Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add an English article "# Install and Understand Byzer in 3 minutes" #298

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
109 changes: 109 additions & 0 deletions Install-and-Understand-Byzer-in-3-minutes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
# Install and Understand Byzer in **3 minutes**

Choose the installation method that works best for your environment:<br>
Install with **Docker**, **Release Package**, and **Desktop Version**.

We strongly recommend that you use the **Release Package** method, because it can be well oriented to Standalone and Yarn-based distribution. Simply follow 3 steps:<br>

1. Download
2. Unzip and modify the configuration (optional)
3. Start service

## Install Byzer Standalone
### Step1: Download
Byzer-lang: https://download.byzer.org/latest/byzer-lang-all-in-one-linux-amd64-3.3.0-2.4.0-SNAPSHOT.tar.gz

Byzer-notebook: https://download.byzer.org/latest/Byzer-Notebook-1.2.3.tar.gz

### Step2: Unzip

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here, we may provide the command how to unzip the downloaded files. e.g.

tar xvf  byzer-lang-all-in-one-linux-amd64-3.3.0-2.4.0-SNAPSHOT.tar.gz

### Step3 Start

Start the Byzer-lang engine

Enter the [byzer-lang-all-in-one-linux-amd64-3.3.0-2.4.0-SNAPSHOT directory](https://download.byzer.org/latest/byzer-lang-all-in-one-linux-amd64-3.3.0-2.4.0-SNAPSHOT.tar.gz), execute the following command to start the Byzer engine:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can try to download this file and extract it and find the really file name. In fact, it's byzer-lang-all-in-one-linux-amd64-3.3.0-2.4.0-SNAPSHOT. There is no need to make this directory name as link.

```
./bin/byzer.sh start
```

After the startup is complete, you can access port 9003 (http://localhost:9003)

To start **Byzer Notebook**, you need to prepare a MySQL database in advance (version 5.7 is recommended), and then create a database named `notebook`.
Now you can enter `Byzer-Notebook-1.2.3`, and modify the `conf/notebook.properties` file, then modify the database configuration part according to the actual address of the database:

```
notebook.database.type=mysql
notebook.database.ip=127.0.0.1
notebook.database.port=3306
notebook.database.name=notebook
notebook.database.username=root
notebook.database.password=root
```
Now it's ready to start **Notebook** by

```
./bin/notebook.sh start
```

Visit port 9002 (http://localhost:9002), and enter the **Notebook** interface to start your project.

## Running the Development Server by **Yarn**

### Step 1:
Download and Unzip spark-3.3.0: https://archive.apache.org/dist/spark/spark-3.3.0/spark-3.3.0-bin-hadoop3.tgz

### Step 2:
Move `core-site.xml`, `yarn-site.xml`, and `hive-site.xml` into the `spark-3.3.0-bin-hadoop3/conf` directory
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy these files from existing Hadoop Cluster to spark-3.3.0-bin-hadoop3/conf directory


### Step 3:
Copy server configuration in directory `byzer-lang-all-in-one-linux-amd64-3.3.0-2.4.0-SNAPSHOT/conf`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here our target is to replace the default byzer.propoerties.overwrite which is for standalone by default.


```
cp byzer.properties.server.example byzer.properties.overwrite
```

Start the engine:
```
export SPARK_HOME=xxxxxxx && ./bin/byzer.sh start
```

> Note: In specific cases, one of the two variables `HADOOP_HOME` or `YARN_CONF_DIR` may also need to be configured.

Now the Byzer engine will run on the Yarn cluster in `yarn-client` mode. You can access the http://localhost:9003 or use it directly through Notebook.

## Deployed in Kubernetes (Beta)
### Step 1
Add repo
```
helm repo add byzer http://store.mlsql.tech/charts
```
### Step 2
Install
```
helm install -n byzer --create-namespace bz byzer/Byzer-lang \
--set clusterUrl=https://192.168.3.42:16443 \
--set fs.defaultFS=oss://xxxx \
--set fs.impl=org.apache.hadoop.fs.aliyun.oss.AliyunOSSFileSystem \
--set fs."oss\.endpoint"=oss-cn-hangzhou.aliyuncs.com \
--set fs."oss\.accessKeyId"=xxxx \
--set fs."oss\.accessKeySecret"=xxxxx
```

> Note: The engine needs an object storage or HDFS. Here it is configured with Alibaba Cloud OSS.

### Step3:
Install **Byzer Notebook** (you need to configure a database and have a library named as `notebook`):):

```
helm install -n byzer --create-namespace nb byzer/Byzer-notebook \
--set name=nb \
--set engine=bz \
--set notebook."database\.ip"=192.168.3.14 \
--set notebook."database\.username"=xxx \
--set notebook."database\.password"=xxxx
```

Congratulations! Now you can access the **Notebook** to see what's available.

For more info, visit https://github.com/byzer-org/byzer-helm