Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Dynamic modification of configuration files&&Slow log #2103

Merged
merged 3 commits into from
Nov 8, 2023
Merged

Dynamic modification of configuration files&&Slow log #2103

merged 3 commits into from
Nov 8, 2023

Conversation

dingxiaoshuai123
Copy link
Collaborator

@dingxiaoshuai123 dingxiaoshuai123 commented Nov 6, 2023 •
edited
Loading

Related issue : #2086
1.Add the Codis config command for dynamic configuration at the Codis level.
2. Add slow logging at the Codis level to record commands that exceed the execution time threshold.

AlexStocks
AlexStocks reviewed Nov 6, 2023
View reviewed changes
codis/pkg/utils/configAux.go Outdated
w := bufio.NewWriter(f)

for _, item := range c.items {
var lineStr string
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

放到for循环外面

@AlexStocks AlexStocks changed the title [wip]Dynamic modification of configuration files. Dynamic modification of configuration files. Nov 7, 2023
AlexStocks
AlexStocks reviewed Nov 7, 2023
View reviewed changes
codis/pkg/proxy/config.go Outdated
@@ -133,6 +133,11 @@ metrics_report_statsd_prefix = ""
`

type Config struct {
ConfigFileName string `toml:"-" json:"config_file_name"`
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 照配置文件中的顺序放置这些成员，不要上来就放到最上面。
2 为何下面三个成员没有 json?

AlexStocks
AlexStocks reviewed Nov 7, 2023
View reviewed changes
codis/pkg/proxy/mapper.go Outdated
@@ -318,3 +320,30 @@ func getHashKey(multi []*redis.Resp, opstr string) []byte {
}
return nil
}

func getWholeCmd(multi []*redis.Resp, cmd []byte) int {
var index = 0
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

var (
index = 0
bytes = 0
)

AlexStocks
AlexStocks reviewed Nov 7, 2023
View reviewed changes
codis/pkg/proxy/mapper.go Outdated
}
bytes += len(multi[i].Value)

// 如果cmd已经满了，那么最后腾出来一个more长度的位置添加more信息
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

改成英文

AlexStocks
AlexStocks reviewed Nov 7, 2023
View reviewed changes
codis/pkg/proxy/proxy.go Outdated
@@ -10,7 +10,9 @@ import (
"os"
"os/exec"
"path/filepath"
"pika/codis/v2/pkg/proxy/redis"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

注意下 import 里面这些包路径的顺序，把这行放到 import 最下面

AlexStocks
AlexStocks reviewed Nov 7, 2023
View reviewed changes
codis/pkg/proxy/proxy.go Outdated Show resolved Hide resolved
AlexStocks
AlexStocks reviewed Nov 7, 2023
View reviewed changes
codis/pkg/proxy/proxy.go Outdated Show resolved Hide resolved
AlexStocks
AlexStocks reviewed Nov 7, 2023
View reviewed changes
codis/pkg/proxy/session.go Outdated
log.Errorf("%s remote:%s, start_time(us):%d, duration(us): [%d, %d, %d], %d, tasksLen:%d",
time.Unix(r.ReceiveTime/1e9, 0).Format("2006-01-02 15:04:05"), s.Conn.RemoteAddr(), r.ReceiveTime/1e3, d0, d1, d2, duration, r.TasksLen)
}
//if s.config.SlowlogLogSlowerThan > 0 {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

为何注释掉？

AlexStocks
AlexStocks reviewed Nov 7, 2023
View reviewed changes
codis/pkg/proxy/session.go Outdated

r.Resp = SlowLogGetByNum(num)
} else if len(r.Multi) == 4 {
var id int64
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more. 

var (
       id int64
	num int64
		 err error

)

AlexStocks
AlexStocks reviewed Nov 7, 2023
View reviewed changes
codis/pkg/proxy/slowlog.go Outdated
PIKA_SLOWLOG_LENGTH_MAX = 10000000
)

type Mutex struct {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这种封装有何意义？

AlexStocks
AlexStocks reviewed Nov 7, 2023
View reviewed changes
codis/pkg/proxy/slowlog.go Outdated
num = int64(PSlowLog.logList.Len())
}
var res = make([]*redis.Resp, 0, num)
var iter = PSlowLog.logList.Front() // 从最新的数据开始
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

后面的注释改成英文

AlexStocks
AlexStocks reviewed Nov 7, 2023
View reviewed changes
codis/pkg/utils/configAux.go Outdated
return err
}

// 拿到一行并去除两边空白字符
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

注释全部改成英文

AlexStocks
AlexStocks reviewed Nov 7, 2023
View reviewed changes
codis/pkg/utils/configAux.go Outdated
for i := 0; i < obj.NumField(); i++ {
fieldInfo := obj.Type().Field(i)
name := fieldInfo.Tag.Get("toml")
// 检查字段的toml tag是否合法
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

改成英文注释

AlexStocks
AlexStocks reviewed Nov 7, 2023
View reviewed changes
codis/pkg/utils/configAux.go Outdated
log.Infof("Show config, len = %d\n", len(c.items))
for index, item := range c.items {
if item.confType == TypeComment {
// 注释的格式: id: context
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

改成英文注释

AlexStocks
AlexStocks reviewed Nov 7, 2023
View reviewed changes
codis/pkg/utils/configAux.go Outdated
// 注释的格式: id: context
log.Infof("%d: %s\n", index, item.name)
} else {
// 配置文件的格式: id: key = value 或者 id: key value
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

改成英文注释

AlexStocks
AlexStocks reviewed Nov 7, 2023
View reviewed changes
codis/pkg/proxy/proxy.go Outdated
}
}

func (s *Proxy) ConfigSet(key, value string) *redis.Resp {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s 统一改成 p

AlexStocks
AlexStocks reviewed Nov 7, 2023
View reviewed changes
codis/pkg/utils/configAux.go Outdated
type DeployConfig struct {
items []*ConfItem
confMap map[string]*ConfItem
sep string //配置项中key、value分隔符
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

英文注释

@dingxiaoshuai123 dingxiaoshuai123 changed the title Dynamic modification of configuration files. Dynamic modification of configuration files&&Slow log Nov 8, 2023
AlexStocks
AlexStocks approved these changes Nov 8, 2023
View reviewed changes
codis/pkg/utils/configAux.go Outdated
if found {
item.value = value
} else {
item := &ConfItem{}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

构造函数内赋值

@AlexStocks AlexStocks merged commit 8ed1123 into OpenAtomFoundation:unstable Nov 8, 2023
11 checks passed
bigdaronlee163 pushed a commit to bigdaronlee163/pika that referenced this pull request Jun 8, 2024
@dingxiaoshuai123
Dynamic modification of configuration files&&Slow log (OpenAtomFounda…
9d8417a 
…tion#2103)

* Dynamic modification of configuration files.Check the correctness of the functionality.

* Modified some specification issues.

* Dynamically modify Codis configuration file and add slow query logs.
cheniujh pushed a commit to cheniujh/pika that referenced this pull request Sep 24, 2024
@dingxiaoshuai123
Dynamic modification of configuration files&&Slow log (OpenAtomFounda…
933b6e8 
…tion#2103)

* Dynamic modification of configuration files.Check the correctness of the functionality.

* Modified some specification issues.

* Dynamically modify Codis configuration file and add slow query logs.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@AlexStocks AlexStocks AlexStocks approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@dingxiaoshuai123 @AlexStocks