Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Embedding] Fix set initialized flag too early in restore subgraph. #920

Merged
merged 1 commit into from
Aug 11, 2023

Conversation

lixy9474
Copy link
Collaborator

@lixy9474 lixy9474 commented Aug 2, 2023

No description provided.

@lixy9474 lixy9474 force-pushed the fix_restore branch 2 times, most recently from 50d11d1 to 27edef2 Compare August 3, 2023 03:40
OP_REQUIRES_OK(c, c->GetAttr("initial_num_buckets", &initial_num_buckets));
is_set_initialized_ = true;
if (initial_num_buckets == embedding::IsSetInitialized::NOT_SET_INITAILIZED)
is_set_initialized_ = false;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

单句也加{ };写个if... else

@@ -406,7 +413,7 @@ class InitializeKvVariableOp : public OpKernel {
core::ScopedUnref unref_me(primary_variable);
}
core::ScopedUnref unref_me(ev);
if (steps_to_live_ != kEmbeddingVarUseDB) {
if (is_set_initialized_) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个if 替换 改变原先逻辑

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个可以参考代码276行,可以保持一致。

@candyzone candyzone changed the title [Embedding] Ensure restore is finished before workers start training. [Embedding] Fix set initialized flag too early in restore subgraph. Aug 7, 2023
@lixy9474 lixy9474 force-pushed the fix_restore branch 2 times, most recently from 397ecf9 to e0b9a85 Compare August 7, 2023 07:23
@candyzone candyzone merged commit 8d8e16a into DeepRec-AI:main Aug 11, 2023
25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants