Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bugfix: 在CMDB跨业务转移主机后从ESB接口立即使用报主机无效 #1909

Closed
jsonwan opened this issue Apr 4, 2023 · 0 comments
Closed
Assignees
Labels
done 已上线到正式环境并验收通过 kind/bug 程序故障Bug,漏洞 stage/prod Production environment in tencent has been deploy

Comments

@jsonwan
Copy link
Collaborator

jsonwan commented Apr 4, 2023

Version / Branch / tag
3.6.x

出了什么问题?(What Happened?)
在CMDB跨业务转移主机后从ESB接口立即使用新业务与主机进行执行报主机无效,在单业务主机数量较大(1w+),总体主机数量较多的环境下偶现。
根据某个案例分析得出的原因:
Job的主机是来源于CMDB的,为了保证主机数据的实时性,Job对CMDB主机的变动事件进行了监听并处理,但事件只是增量数据,为了保证数据的完整性,也会定时按业务从CMDB全量同步主机数据,问题出现的情况是在20:40:16收到了转移到B业务的事件并更新了主机信息,但是同时正好有一个同步A业务下主机的线程使用20:38:40的主机数据(此时主机还在A业务下)在20:40:46更新到DB了,导致旧数据覆盖了新数据,所以在后面的调用中主机被判定在A业务下了。
解决方案:
对判定为在其他业务下的主机使用CMDB实时数据进行二次判定,在更细的粒度上应用“缓存未命中则从数据源查询”的逻辑。

如何复现?(How to reproduce?)
在CMDB跨业务转移主机,完成后从ESB接口立即使用新业务与主机进行执行,有一定概率报主机无效。

预期结果(What you expect?)
正常执行。

@jsonwan jsonwan added the kind/bug 程序故障Bug,漏洞 label Apr 4, 2023
@jsonwan jsonwan self-assigned this Apr 4, 2023
jsonwan added a commit to jsonwan/bk-job that referenced this issue Apr 4, 2023
在其他业务下的主机根据CMDB实时数据进行二次判定
jsonwan added a commit to jsonwan/bk-job that referenced this issue Apr 4, 2023
在其他业务下的主机根据CMDB实时数据进行二次判定
jsonwan added a commit to jsonwan/bk-job that referenced this issue Apr 6, 2023
jsonwan added a commit that referenced this issue Apr 6, 2023
bugfix: 在CMDB跨业务转移主机后从ESB接口立即使用报主机无效 #1909
jsonwan added a commit to jsonwan/bk-job that referenced this issue Apr 6, 2023
在其他业务下的主机根据CMDB实时数据进行二次判定
jsonwan added a commit that referenced this issue Apr 6, 2023
bugfix: 在CMDB跨业务转移主机后从ESB接口立即使用报主机无效 #1909
@bkjob-bot bkjob-bot added the stage/prod Production environment in tencent has been deploy label Apr 13, 2023
@bkjob-bot bkjob-bot added the done 已上线到正式环境并验收通过 label May 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
done 已上线到正式环境并验收通过 kind/bug 程序故障Bug,漏洞 stage/prod Production environment in tencent has been deploy
Projects
None yet
Development

No branches or pull requests

2 participants