Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Trigger sync compaction for bookmarks every week #3749

Merged
merged 5 commits into from
Nov 7, 2019
Merged

Conversation

darkdh
Copy link
Member

@darkdh darkdh commented Oct 18, 2019

fix brave/brave-browser#6552

Submitter Checklist:

Test Plan:

#3749 (comment) can be referenced to generate update records easily

Update

  1. Make sure launch old Brave on device A and device B for fresh profiles
  2. Sync device A and device B
  3. Sync 2 bookmarks
  4. Keep changing bookmarks name or URL of the 2 bookmarks for multiple times on device A
  5. Make sure device B get all the update
  6. Quit Brave on device A and launch new Brave with the same profile
  7. Open chrome://inspect/#extensions and inspect Brave Sync on device A
  8. After seeing compaction deletes ... on console and wait for extra 3 mins
  9. Connect device C to the sync chain
  10. Open chrome://inspect/#extensions and inspect Brave Sync on device C
  11. You should see exactly got 2 decrypted records in BOOKMARKS after 0 on console
  12. And 2 bookmarks are up to date with device A and device B

Delete

  1. Make sure launch old Brave on device A and device B for fresh profiles
  2. Sync device A and device B
  3. Sync 1 bookmark
  4. Keep changing bookmarks name or URL of the bookmark for multiple times on device A
  5. Make sure device B get all the update
  6. Delete the bookmark on device A
  7. Make sure device B also deletes the bookmark
  8. Quit Brave on device A and launch new Brave with the same profile
  9. Open chrome://inspect/#extensions and inspect Brave Sync on device A
  10. After seeing compaction deletes ... on console and wait for extra 3 mins
  11. Connect device C to the sync chain
  12. Open chrome://inspect/#extensions and inspect Brave Sync on device C
  13. You should see exactly got 1 decrypted records in BOOKMARKS after 0 on console
  14. And device C should still have no bookmark

Reviewer Checklist:

  • New files have MPL-2.0 license header.
  • Request a security/privacy review as needed.
  • Adequate test coverage exists to prevent regressions
  • Verify test plan is specified in PR before merging to source

After-merge Checklist:

  • The associated issue milestone is set to the smallest version that the
    changes has landed on.
  • All relevant documentation has been updated.

@darkdh darkdh self-assigned this Oct 18, 2019
Copy link
Contributor

@AlexeyBarabash AlexeyBarabash left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR looks good

@AlexeyBarabash
Copy link
Contributor

AlexeyBarabash commented Oct 22, 2019

Verified it works with STR of:

I. Making bookmarks with lots of updates

  1. Get clean profiles A and B
  2. Establish the sync chain, wait until there will be two devices
  3. Open page brave://bookmarks on device A
  4. Call page inspector, right click on tree side => Inspect
  5. On console tab paste the code
for (var i = 0; i < 10; ++i) {
chrome.bookmarks.create({'parentId':BOOKMARKS_BAR_ID, 'title': `Bookmark_${i}`,
                         'url': `https://A_${i}.com`});
}

  1. Wait while bookmarks will appear on device B
  2. On device A paste and enter this in brave://bookmarks Inspector console:
function processNode(node) {
    if(node.children) {
        node.children.forEach(function(child) { processNode(child); });
    }
    if(node.title && node.url) { 
	chrome.bookmarks.update(node.id, {'title': node.title+`_A`});
    }
}
chrome.bookmarks.getTree(function(itemTree){
    itemTree.forEach(function(item){
        processNode(item);
    });
});

paste and run it several times
8. Wait for device B will get the changes
9. Close browser device A

II. Initiating compact operation

  1. Open folder of profile A, find Preferences file, in my case this is /home/alexey/.config/BraveSoftware/TEST/A/Default/Preferences
  2. Open Preferences file with text editor and remove part like "last_compact_time":"13216219229248049", , save and close file
  3. Launch device A
  4. Ensure you see compaction deletes messages in console or in inspector console

III. Verifying compact was correct

  1. Get clean profile C
  2. Copy sync code from device A
  3. Connect device C to the sync chain
  4. Ensure device C has the same bookmarks as device A
  5. See in inspector console for Brave Sync extension get-existing-objects gives objects without repeating by objectId

extensions::api::brave_sync::OnSendCompactSyncCategory::Create(
category_name).release());
std::unique_ptr<Event> event(
new Event(extensions::events::FOR_TEST,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FOR_TEST?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's been this way since we have BraveSyncEventRouter

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is just a events::HistogramValue and I can't find a suitable value from the existing list. Should we extend the enum? Please note that brave sync will eventually go away and migrated to native implementation

base::Time::Now() - last_compact_time >
base::TimeDelta::FromDays(kCompactPeriodInDays)) {
brave_sync_client_->SendCompactSyncCategory(kBookmarks);
brave_sync_prefs_->SetLastCompactTime(base::Time::Now());
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't we wait to do this until the compaction is complete?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You mean, when the objects are really deleted on s3?
The compaction API is designed to have no callback, if we want callback when the object is really deleted, we would need extra listObject query to check. The deletion is not instant, it takes approximately 1 minute

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, the compaction could take a significant long time to pull down all the record if the recordset is huge

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

addressed in bb0c6c3
which requires deps update when brave/sync#355 is merged

Copy link
Member

@SergeyZhukovsky SergeyZhukovsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

++

@darkdh darkdh changed the title Triger sync compaction for bookmarks every week Trigger sync compaction for bookmarks every week Oct 22, 2019
@darkdh darkdh force-pushed the sync-compaction branch 2 times, most recently from 56d2773 to aef2165 Compare October 29, 2019 02:38
@darkdh darkdh force-pushed the sync-compaction branch 2 times, most recently from bda5e45 to 527100a Compare November 6, 2019 20:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement client side changes of sync compaction
4 participants