diff --git a/docs/network-flow-visibility.md b/docs/network-flow-visibility.md index d41210c93b5..3751c371233 100644 --- a/docs/network-flow-visibility.md +++ b/docs/network-flow-visibility.md @@ -659,8 +659,8 @@ kubectl delete -f https://raw.githubusercontent.com/Altinity/clickhouse-operator ##### Credentials Configuration -ClickHouse credentials are specified in [clickhouse.yml][clickhouse_manifest_yaml] as -a resource of kind: a Secret. If the username `clickhouse_operator` has changed, please +ClickHouse credentials are specified in `flow-visibility.yml` as a resource of name: +`clickhouse-secret`. If the username `clickhouse_operator` has been changed, please update the following section accordingly. ```yaml @@ -678,26 +678,21 @@ spec: clickhouse_operator/networks/ip: "::/0" ``` -ClickHouse credentials are also specified in [flow-aggregator.yml][flow_aggregator_manifest_yaml] -as a resource of kind: a Secret. Please also make the corresponding changes. +ClickHouse credentials are also specified in `flow-aggregator.yml` as a resource +of name: `clickhouse-secret`. Please also make the corresponding changes. -Grafana login credentials are specified in [grafana.yml][grafana_manifest_yaml] as -resource of kind: a Secret. +Grafana login credentials are specified in `flow-visibility.yml` as a resource of +name: `grafana-secret`. -We recommend changing all the credentials above if you are going to run the Flow Collector -in production. After making any credentials change, run the following command to generate -a new manifest: - -```shell -make manifest -``` +We recommend changing all the credentials above if you are going to run the Flow +Collector in production. ##### ClickHouse Configuration The ClickHouse database can be accessed through the service `clickhouse-clickhouse`. The pod exposes HTTP port at 8123 and TCP port at 9000 by default. The ports are -specified in [clickhouse.yml][clickhouse_manifest_yaml] as `serviceTemplates`. -To use other ports, please update the following section accordingly. +specified in `flow-visibility.yml` as `serviceTemplates` of a resource of kind: +a `ClickHouseInstallation`. To use other ports, please update the following section. ```yaml serviceTemplates: @@ -710,34 +705,53 @@ serviceTemplates: port: 9000 ``` -This service is also used by the Flow Aggregator and Grafana. If you update the -HTTP port, please update `url` in [datasource_provider.yml][grafana_datasouce_provider_yaml]. -If you update the TCP port, please update `jsonData.port` in [datasource_provider.yml][grafana_datasouce_provider_yaml] -and `databaseURL` in the [Flow Aggregator Configuration](#configuration-1). +This service is used by the Flow Aggregator and Grafana. + +- If you have changed the HTTP port, please update the `url` of a resource of name `grafana-datasource-provider` in `flow-visibility.yml`. + +- If you have changed the TCP port, please update the `databaseURL` following [Flow Aggregator Configuration](#configuration-1), +and also update the `jsonData.port` of the `grafana-datasource-provider` resource. + +```yaml +apiVersion: v1 +data: + datasource_provider.yml: | + apiVersion: 1 + datasources: + - name: ClickHouse + type: grafana-clickhouse-datasource + access: proxy + url: http://clickhouse-clickhouse.flow-visibility.svc:8123 + editable: true + jsonData: + server: clickhouse-clickhouse.flow-visibility.svc + port: 9000 + username: $CLICKHOUSE_USERNAME + secureJsonData: + password: $CLICKHOUSE_PASSWORD +kind: ConfigMap +metadata: + name: grafana-datasource-provider-h868k56k95 + namespace: flow-visibility +``` The ClickHouse throughput depends on two factors - the storage size of the ClickHouse and the time interval between the batch commits to the ClickHouse. Larger storage size and longer commit interval provide higher throughput. Grafana flow collector supports the ClickHouse in-memory deployment with limited -storage size. This is specified in [clickhouse.yml][clickhouse_manifest_yaml]. -The default value of storage size for the ClickHouse server is 8 GiB. Users +storage size. This is specified in `flow-visibility.yml` under the `ClickHouseInstallation` +resource. The default value of storage size for the ClickHouse server is 8 GiB. Users can expect a linear growth in the ClickHouse throughput when they enlarge the storage size. For development or testing environment, you can decrease the storage size to 2GB. To deploy the ClickHouse with a different storage size, please modify the `sizeLimit` in the following section. ```yaml -- name: clickhouse-storage-volume - emptyDir: +- emptyDir: medium: Memory sizeLimit: 8Gi -``` - -After making the change, run the following command to generate a new manifest: - -```shell -./hack/generate-manifest-flow-visibility.sh > build/yamls/flow-visibility.yml + name: clickhouse-storage-volume ``` The time interval between the batch commits to the ClickHouse is specified in the @@ -831,8 +845,9 @@ Currently we only support the visualization of NetworkPolicies with `Allow` acti If you would like to make any changes to any of the pre-built dashboards, or build a new dashboard, please follow this [doc](https://grafana.com/docs/grafana/latest/dashboards/export-import/) -for dashboard export and import. To generate a deployment manifest with the changes, -please follow the following steps: +for dashboard JSON files export and manual import. + +To generate a deployment manifest with the changes, please follow the following steps: 1. Clone the repository. Exported dashboard JSON files should be placed under `antrea/build/yamls/flow-visibility/base/provisioning/dashboards`. 1. If a new dashboard is added, edit [kustomization.yml][flow_visibility_kustomization_yaml] @@ -1017,8 +1032,4 @@ With filters applied: Flow Visualization Network Policy Dashboard -[clickhouse_manifest_yaml]: ../build/yamls/flow-visibility/base/clickhouse.yml -[flow_aggregator_manifest_yaml]: ../build/yamls/flow-aggregator/base/flow-aggregator.yml -[grafana_manifest_yaml]: ../build/yamls/flow-visibility/base/grafana.yml -[grafana_datasouce_provider_yaml]: ../build/yamls/flow-visibility/base/provisioning/datasources/datasource_provider.yml [flow_visibility_kustomization_yaml]: ../build/yamls/flow-visibility/base/kustomization.yml