-
Notifications
You must be signed in to change notification settings - Fork 67
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[Doc] add load_data_using_stream_load.md
Signed-off-by: yandongxiao <[email protected]>
- Loading branch information
1 parent
95436e2
commit 38ebcc9
Showing
8 changed files
with
853 additions
and
2 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Large diffs are not rendered by default.
Oops, something went wrong.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,88 @@ | ||
# Load Data Using Stream Load | ||
|
||
The issue is that when the load client residents in a different network other than FE/BE's private network. FE's HTTP | ||
307 brings BE's private network address to the client who does not recognize and can't process the redirection. | ||
|
||
## Issues of Stream Load | ||
|
||
Stream Load Interaction: | ||
|
||
![Stream Load Interaction](./images/stream_load_interaction.png) | ||
|
||
The issues: | ||
|
||
![Issues When Client Is Not In the Same Private Network](./images/issues_of_stream_load.png) | ||
|
||
## Issues of Transactional Stream Load | ||
|
||
Transactional Stream Load Interaction: | ||
|
||
![Transactional Stream Load Interaction](./images/transactional_stream_load_interaction.png) | ||
|
||
The issues: | ||
|
||
![Issues When Client Is Not In the Same Private Network](./images/issues_of_transactional_stream_load.png) | ||
|
||
## Using Fe Proxy to Solve the Issues | ||
|
||
![Using Fe Proxy to Solve the Issues](./images/stream_load-fe-proxy.png) | ||
|
||
## How to Deploy Fe Proxy | ||
|
||
### Deploy Fe Proxy Using Helm | ||
|
||
If you install StarRocks with Helm, you can add the following configuration to the `values.yaml` file: | ||
|
||
For `kube-starrocks` Helm chart: | ||
|
||
```yaml | ||
starrocks: | ||
starRocksFeProxySpec: | ||
replicas: 1 | ||
# set the resolver for nginx server, default kube-dns.kube-system.svc.cluster.local | ||
resolver: "" | ||
limits: | ||
cpu: 1 | ||
memory: 2Gi | ||
requests: | ||
cpu: 1 | ||
memory: 2Gi | ||
service: | ||
type: NodePort | ||
ports: | ||
- containerPort: 8080 | ||
name: http-port | ||
nodePort: 30180 | ||
port: 8080 | ||
``` | ||
For `starrocks` Helm chart: | ||
|
||
```yaml | ||
starRocksFeProxySpec: | ||
replicas: 1 | ||
# set the resolver for nginx server, default kube-dns.kube-system.svc.cluster.local | ||
resolver: "" | ||
limits: | ||
cpu: 1 | ||
memory: 2Gi | ||
requests: | ||
cpu: 1 | ||
memory: 2Gi | ||
service: | ||
type: NodePort | ||
ports: | ||
- containerPort: 8080 | ||
name: http-port | ||
nodePort: 30180 | ||
port: 8080 | ||
``` | ||
|
||
Please | ||
see https://github.com/StarRocks/starrocks-kubernetes-operator/blob/main/helm-charts/charts/kube-starrocks/values.yaml | ||
for more details about how to configure `starRocksFeProxySpec`. | ||
|
||
### Deploy Fe Proxy Using Helm | ||
|
||
If you install StarRocks with StarRocksCluster CR yaml, please see [deploy_a_starrocks_cluster_with_fe_proxy.md]( | ||
../examples/starrocks/deploy_a_starrocks_cluster_with_fe_proxy.yaml) |