Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Replace cb-store with etcd #1695

Merged
merged 26 commits into from
Jul 30, 2024
Merged

Replace cb-store with etcd #1695

merged 26 commits into from
Jul 30, 2024

Conversation

yunkon-kim
Copy link
Member

This PR will replace cb-strelog with kvstore based on etcd.

  • kvstore, kvutil, etcd pkgs will be added.
  • docker compose will run etcd server and configure a basic level auth to etcd.
  • There are two methods to use docker compose.

Please see below for further details.

yunkon-kim and others added 24 commits July 25, 2024 22:42
@yunkon-kim
Add etcd auth
d707e85 
* Add `setup-auth.sh` and `.env`, which is used by Docker Compose
* Set up or apply auth on etcd server through `etcd-setup-auth` (an ephemeral container)
* Improve configuration for connecting to `etcd` and `kvstore`
  - Add environment variables of `etcd`
  - Read envs and set to the configs (including auth info)
  - Retry several times until you connect to the `etcd` server
* Update examples for `etcd` test
@yunkon-kim
Exclude key not found from errors
bd8a13f
@yunkon-kim
Replace cb-store with kvstore
b4973c4
@yunkon-kim
Relocate etcd and kvstore initialization section
f9ae7b3
@yunkon-kim
Update configs and dependencies
93b1c11
@yunkon-kim
Update README.md
c333010
@cb-github-robot
Add etcd-centric kvstore and replace cb-store
74ac071
@yunkon-kim
Add and update docker-compose configuration to customize or set envs
9dfb368 
* Add `.env` and set variables with interpolation in the compose file
* Specify environment variables for Docker-Compose
* Match the specified environment variables with the environment variables of each service
* Set default values ​​for each service environment variable
@yunkon-kim
Ignore docker-compose files and .env files
edfbd08
@yunkon-kim
Fix env name and disable cb-tumblebug build
9ab10ba
@cb-github-robot
Add and update docker-compose configuration to customize or set envir…
b28bc3f 
…onment variables
@yunkon-kim
Add etcd auth
1c20f63 
* Add `setup-auth.sh` and `.env`, which is used by Docker Compose
* Set up or apply auth on etcd server through `etcd-setup-auth` (an ephemeral container)
* Improve configuration for connecting to `etcd` and `kvstore`
  - Add environment variables of `etcd`
  - Read envs and set to the configs (including auth info)
  - Retry several times until you connect to the `etcd` server
* Update examples for `etcd` test
@yunkon-kim
Exclude key not found from errors
ce10210
@yunkon-kim
Replace cb-store with kvstore
471844f
@yunkon-kim
Relocate etcd and kvstore initialization section
a3980fc
@yunkon-kim
Update configs and dependencies
39989e5
@yunkon-kim
Update README.md
e7a0670
@yunkon-kim
Add and update docker-compose configuration to customize or set envs
cb23136 
* Add `.env` and set variables with interpolation in the compose file
* Specify environment variables for Docker-Compose
* Match the specified environment variables with the environment variables of each service
* Set default values ​​for each service environment variable
@yunkon-kim
Ignore docker-compose files and .env files
fc500b7
@yunkon-kim
Fix env name and disable cb-tumblebug build
6279ae7
@yunkon-kim
Merge branch 'replace-with-etcd' into 240730-09
57c71f0
@cb-github-robot
Rebase replace-with-etcd onto upstream latest main
ad90808
@yunkon-kim
Finalize docker-compose and etcd
79c3ebd 
* Add environment variables of etcd to `setup.env`, `Dockerfile`, `docker-compose.yaml`, and `docker-compose-custom.yaml`
* Specify `pull_policy: missing` for cb-tumblebug service in `docker-compose.yaml` and `docker-compose-custom.yaml`
* Read `etcd` environment variables and initialize `kvstore`
* Note - the endpoints environment variable of TB in `.env` specifies the target, because setting the environment variable can affect the relationship between compose services.
@cb-github-robot
Finalize docker-compose and etcd
457c16c
@github-actions github-actions bot added the docs Improvements or additions to documentation label Jul 30, 2024
@yunkon-kim yunkon-kim self-assigned this Jul 30, 2024
@seokho-son
Copy link
Member

@yunkon-kim

scripts/cleanDB.sh

https://github.com/cloud-barista/cb-tumblebug/blob/main/scripts/cleanDB.sh#L13

도 기존대로 동작하려나요? 아님, container-volume 전체를 삭제하는게 바람직하려나요.

@yunkon-kim
Copy link
Member Author

@seokho-son

etcd의 경우 volume을 다음과 같이 설정해 둔 상황으로 기존대로 동작하지 않을 것으로 보입니다.

volumes: 
      - ./container-volume/etcd/data:/etcd-data

확인해보니 TBMETAPATH="$CBTUMBLEBUG_ROOT/meta_db/dat"에서 sqlite 관련 데이터가 저장되고 있는 것으로 확인했습니다.

별도의 etcd 데이터 삭제 구문을 추가하도록 하겠습니다.

@yunkon-kim
Copy link
Member Author

@seokho-son

meta_db/dat 저장 경로가 두 군데 인 것 같습니다. (소스 기반 실행 시, 컨테이너 실행 시)

cb-tumblebug/scripts/cleanDB.sh

Line 13 in 28d86a7

TBMETAPATH="$CBTUMBLEBUG_ROOT/meta_db/dat"

cb-tumblebug/scripts/runContainer.sh

Line 19 in 28d86a7

CONTAINER_VOLUME_PATH="$CBTUMBLEBUG_ROOT/container-volume/${CONTAINER_NAME}-container"

어떻게 처리하면 좋을지 문의드립니다. (cleanDB.sh 실행 시 둘 다 삭제?)

@seokho-son
Copy link
Member

@seokho-son

meta_db/dat 저장 경로가 두 군데 인 것 같습니다. (소스 기반 실행 시, 컨테이너 실행 시)

cb-tumblebug/scripts/cleanDB.sh

Line 13 in 28d86a7

TBMETAPATH="$CBTUMBLEBUG_ROOT/meta_db/dat"

cb-tumblebug/scripts/runContainer.sh

Line 19 in 28d86a7

CONTAINER_VOLUME_PATH="$CBTUMBLEBUG_ROOT/container-volume/${CONTAINER_NAME}-container"

어떻게 처리하면 좋을지 문의드립니다. (cleanDB.sh 실행 시 둘 다 삭제?)

넵, 둘 다 삭제하는 것이 바람직해보입니다. :)

seokho-son
seokho-son approved these changes Jul 30, 2024
View reviewed changes
Copy link
Member

@seokho-son seokho-son left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@seokho-son
Copy link
Member

/approve

@github-actions github-actions bot added the approved This PR is approved and will be merged soon. label Jul 30, 2024
@cb-github-robot cb-github-robot merged commit f83b325 into main Jul 30, 2024
3 of 4 checks passed
@cb-github-robot cb-github-robot deleted the replace-with-etcd branch July 30, 2024 09:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@seokho-son seokho-son seokho-son approved these changes

@github-actions GitHub Actions GitHub Actions approved these changes

Assignees

@yunkon-kim yunkon-kim

Labels
approved This PR is approved and will be merged soon. docs Improvements or additions to documentation
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@yunkon-kim @seokho-son @cb-github-robot