Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"Last sync was incomplete" message, synchronization via WebDAV not possible #3641

Open
codeling opened this issue Oct 31, 2024 · 7 comments
Open
Labels

Comments

@codeling
Copy link
Contributor

codeling commented Oct 31, 2024

I have searched for similar issues, but haven't found one exactly describing my issue:

Your Environment

  • Version used: Desktop version from Setup.exe, 10.0.11 (on both "good" and "problematic" machine)
  • Operating System and version: Windows 11 (recently upgraded, but also happened before on Windows 10)

Expected Behavior

I can synchronize the superproductivity state between two machines.

Current Behavior

On the "problematic" machine, triggering the sync causes this message to show up (at least it does now after a fresh install/ cleaning of all superproductivity data):
superproductivity-conflict

When I press "Keep remote", I get this message:
superproductivity-incomplete

As superproductivity seems unable to read the data it loaded, there is no way to continue sync...

Steps to Reproduce (for bugs)

For the "problematic" machine where the bug happens:

  1. Install superproductivity
  2. Set up sync with the exact same parameters as on the other machine (sync provider: WebDAV; URL on a nextcloud; same subfolder as on working machine; username with an app password; I tried both compression on and off; the synchronized file size changes, but not the strange (chinese?) letter output in the console on the affected machine (see below).
  3. Synchronize - and see the messages shown above

Can you reproduce this reliably?

This happens every time I try to sync from this one machine (sync on the "good" machine seems to work fine, at least there's no error messages there, and the data on the server seems valid json).
I even tried to completely remove (uninstall + remove any files in AppData/Roaming/superProductivity); when setting the sync up again -> exactly same issue.

I have tried with compression on and off; and even though the served file only looks like as shown in the console log with compression turned on, in super-productivity, I see the strange letters on the "problematic" machine no matter whether compression is turned on or off. I have verified that the MAIN.json file on the server actually contains readable text instead of the gibberish shown in the log, when compression is turned off and sync is triggered on the "working" machine.

Maybe some transfer compression gets in the way? But why then does this only happen on one machine?

I don't think it makes a difference but the problematic machine is in the same network as the server, while the machine where it works is remote.

Console Output

LOAD COMPLETE false
persistence-local.service.ts:40 {Dropbox: {…}, WebDAV: {…}, LocalFile: {…}}
persistence.service.ts:313 LOAD COMPLETE false
web-dav-api.service.ts:175 {r: {…}}
web-dav-sync.service.ts:131 cleaned rev e4f1173ccd15e943eebac57f1cf22cf9
sync-provider.service.ts:246 {rev: 'e4f1173ccd15e943eebac57f1cf22cf9', localRev: null}
persistence.service.ts:313 LOAD COMPLETE false
sync-provider.service.ts:986 WebDAV ↓ downloading main file
persistence-local.service.ts:40 {Dropbox: {…}, WebDAV: {…}, LocalFile: {…}}
web-dav-api.service.ts:175 {r: {…}}
web-dav-sync.service.ts:131 cleaned rev e4f1173ccd15e943eebac57f1cf22cf9-gzip
sync-provider.service.ts:882 dd 1
sync-provider.service.ts:892 dd 2
sync-provider.service.ts:905 dd 3
check-for-update.util.ts:69 {local: 1730357178052, lastSync: 0, remote: 1730357137738}
check-for-update.util.ts:81 {lastSync: 0, remote: 1730357137.738, local: 1730357178.052} 1730357418.178
check-for-update.util.ts:42 DATA DIVERGED: local > remote
sync-provider.service.ts:986 WebDAV ^--------^-------^
sync-provider.service.ts:986 WebDAV ⇎ X Diverged Data
sync-provider.service.ts:986 WebDAV Dialog => ↓ Update Local
sync-provider.service.ts:986 WebDAV Archive force update on local chosen. Downloading...
sync-provider.service.ts:986 WebDAV ↓ downloading archive file
persistence-local.service.ts:40 {Dropbox: {…}, WebDAV: {…}, LocalFile: {…}}
web-dav-api.service.ts:175 {r: {…}}
web-dav-sync.service.ts:131 cleaned rev 85f911d2d6f82892443b11f565d6ff02-gzip
sync-provider.service.ts:882 dd 1
sync-provider.service.ts:892 dd 2
sync-provider.service.ts:905 dd 3
sync-provider.service.ts:939 SyntaxError: Unexpected token 'ᯡ', "ᯡࠫ䄬ഀ欤ࠩ悃LӐ෠"... is not valid JSON
    at JSON.parse (<anonymous>)
    at e.<anonymous> (sync-provider.service.ts:937:23)
    at Generator.next (<anonymous>)
    at o (main-7WUP32YU.js:1:1557)
    at a.invoke (zone.js:331:158)
    at Object.onInvoke (core.mjs:7029:25)
    at a.invoke (zone.js:331:46)
    at b.run (zone.js:111:37)
    at zone.js:2366:30
    at a.invokeTask (zone.js:356:171)
(anonymous) @ sync-provider.service.ts:939
o @ main-7WUP32YU.js:1
invoke @ zone.js:331
onInvoke @ core.mjs:7029
invoke @ zone.js:331
run @ zone.js:111
(anonymous) @ zone.js:2366
invokeTask @ zone.js:356
onInvokeTask @ core.mjs:7018
invokeTask @ zone.js:356
runTask @ zone.js:157
Z @ zone.js:527
invokeTask @ zone.js:436
_ @ zone.js:1054
x @ zone.js:1084
K @ zone.js:1115
XMLHttpRequest.send
O @ zone.js:2040
scheduleTask @ zone.js:346
onScheduleTask @ zone.js:257
scheduleTask @ zone.js:342
scheduleTask @ zone.js:196
scheduleMacroTask @ zone.js:218
Yn @ zone.js:647
(anonymous) @ zone.js:2078
n.<computed> @ zone.js:929
(anonymous) @ index.js:78
N @ zone.js:2529
n.exports @ index.js:24
n.exports @ index.js:335
y.request @ index.js:217
(anonymous) @ index.js:519
(anonymous) @ index.js:3645
(anonymous) @ index.js:2029
h.execute @ index.js:2115
h.patchInline @ index.js:2144
Ve @ index.js:3644
ke @ index.js:3665
(anonymous) @ index.js:4216
(anonymous) @ index.js:4207
customRequest @ index.js:4733
(anonymous) @ web-dav-api.service.ts:174
o @ main-7WUP32YU.js:1
invoke @ zone.js:331
onInvoke @ core.mjs:7029
invoke @ zone.js:331
run @ zone.js:111
(anonymous) @ zone.js:2366
invokeTask @ zone.js:356
onInvokeTask @ core.mjs:7018
invokeTask @ zone.js:356
runTask @ zone.js:157
Z @ zone.js:527
invokeTask @ zone.js:436
_ @ zone.js:1054
x @ zone.js:1084
K @ zone.js:1115
XMLHttpRequest.send
O @ zone.js:2040
scheduleTask @ zone.js:346
onScheduleTask @ zone.js:257
scheduleTask @ zone.js:342
scheduleTask @ zone.js:196
scheduleMacroTask @ zone.js:218
Yn @ zone.js:647
(anonymous) @ zone.js:2078
n.<computed> @ zone.js:929
(anonymous) @ index.js:78
N @ zone.js:2529
n.exports @ index.js:24
n.exports @ index.js:335
y.request @ index.js:217
(anonymous) @ index.js:519
(anonymous) @ index.js:3645
(anonymous) @ index.js:2029
h.execute @ index.js:2115
h.patchInline @ index.js:2144
Ve @ index.js:3644
ke @ index.js:3665
(anonymous) @ index.js:4331
(anonymous) @ index.js:4317
(anonymous) @ index.js:4358
(anonymous) @ index.js:4317
getFileContents @ index.js:4745
(anonymous) @ web-dav-api.service.ts:208
o @ main-7WUP32YU.js:1
invoke @ zone.js:331
onInvoke @ core.mjs:7029
invoke @ zone.js:331
run @ zone.js:111
(anonymous) @ zone.js:2366
invokeTask @ zone.js:356
onInvokeTask @ core.mjs:7018
invokeTask @ zone.js:356
runTask @ zone.js:157
Z @ zone.js:527
invokeTask @ zone.js:436
_ @ zone.js:1054
x @ zone.js:1084
K @ zone.js:1115
IndexedDB
(anonymous) @ index.js:107
(anonymous) @ index.js:212
(anonymous) @ main-7WUP32YU.js:1
N @ zone.js:2529
Te @ main-7WUP32YU.js:1
o @ index.js:202
(anonymous) @ indexed-db-adapter.service.ts:68
o @ main-7WUP32YU.js:1
invoke @ zone.js:331
onInvoke @ core.mjs:7029
invoke @ zone.js:331
run @ zone.js:111
(anonymous) @ zone.js:2366
invokeTask @ zone.js:356
onInvokeTask @ core.mjs:7018
invokeTask @ zone.js:356
runTask @ zone.js:157
Z @ zone.js:527
invokeTask @ zone.js:436
E.useG.invoke @ zone.js:422
g.args.<computed> @ zone.js:1687
setTimeout
c @ zone.js:1689
scheduleTask @ zone.js:346
onScheduleTask @ zone.js:257
scheduleTask @ zone.js:342
scheduleTask @ zone.js:196
scheduleMacroTask @ zone.js:218
Yn @ zone.js:647
(anonymous) @ zone.js:1727
n.<computed> @ zone.js:929
_waitForAnimationToComplete @ dialog.mjs:209
_startExitAnimation @ dialog.mjs:169
close @ dialog.mjs:388
close @ dialog-sync-conflict.component.ts:39
(anonymous) @ dialog-sync-conflict.component.html:38
Wte @ core.mjs:25806
n @ core.mjs:25838
(anonymous) @ platform-browser.mjs:679
invokeTask @ zone.js:356
onInvokeTask @ core.mjs:7018
invokeTask @ zone.js:356
runTask @ zone.js:157
invokeTask @ zone.js:433
_ @ zone.js:1054
x @ zone.js:1084
K @ zone.js:1115
Show 132 more frames
Show less
sync-provider.service.ts:940 dataInStr ᯡࠫ䄬ഀ欤ࠩ悃LӐ෠匰תᨠ䳃恖䐠ఠᐧ㠡烀㌠⼠ᨠ䑠͢�®ÖᑸĴ夤Ǔ㢠ށ値ቨာ悠ౠզ⁋ᙠ⒠୚䁋ᐠ唠㤹�嘡Ḡᰡᬚƥ眤晜ຉဩ‫‥࣋䲄l暝C+ ㇐ũ抄Ò�㽏፦怣㙩壪̜䢶㩫ᨪ٠Ŏ〡䡕梍ᧂ䙻^䰡倯ဦ䥱㉨ょ⭓垳峨ㆶҀż唚␸暺➸Ჹ䠤猔፥㘚ᤤ#䅎䤀ϔі碄甜䚇⒬ぽⶓ᠈ι႞☨៉㄀㰣彯㏐湺䨥焐�ર⇲ೂ㪪ㆲⱴ⢳Ꭲᰠᇋء䦄䌩䕥ト糂濦禊奒⳧砺惃䍀䄢嶩䧇⡐ᢢ⍄᥄⮯挤ヮ儬代儂భ䜲䈌第䚧䐒碐⢁䒞嶬䃄Ԉ晚ⰵ啔ሯ໡䕚纬૦▲敯ඓ�぀⚥泮罃掄樌惁ا⣼ᥝ䚔в獚⌣儢᥇⥀䐘῟ে抄ప䫧ᨒ䴶௴炘䉞䣖ヮ䉏ࢆㆲᤢ▦㦦儦࠶ኘ䗃䒦㨚ᵂ
sync-provider.service.ts:950 dd 4
sync-provider.service.ts:676 {remoteArchiveRev: '85f911d2d6f82892443b11f565d6ff02-gzip', remoteMainFileData.archiveRev: '85f911d2d6f82892443b11f565d6ff02', remoteArchiveRevD: Invalid Date, remoteMainFileData.archiveRevD: Invalid Date}
global-progress-bar.service.ts:35 Global spinner was shown forever (30s). Forcing countDown!
(anonymous) @ global-progress-bar.service.ts:35
_next @ tap.js:39
next @ Subscriber.js:50
XRe @ timer.js:30
_execute @ AsyncAction.js:50
execute @ AsyncAction.js:39
flush @ AsyncScheduler.js:33
w.<computed> @ zone.js:1705
invokeTask @ zone.js:356
onInvokeTask @ core.mjs:7018
invokeTask @ zone.js:356
runTask @ zone.js:157
invokeTask @ zone.js:433
E.useG.invoke @ zone.js:422
g.args.<computed> @ zone.js:1687
setInterval
c @ zone.js:1689
scheduleTask @ zone.js:346
onScheduleTask @ zone.js:257
scheduleTask @ zone.js:342
scheduleTask @ zone.js:196
scheduleMacroTask @ zone.js:218
Yn @ zone.js:647
(anonymous) @ zone.js:1727
n.<computed> @ zone.js:929
requestAsyncId @ AsyncAction.js:25
schedule @ AsyncAction.js:21
schedule @ Scheduler.ts:67
schedule @ AsyncScheduler.js:19
(anonymous) @ timer.js:17
_trySubscribe @ Observable.js:42
subscribe @ Observable.js:28
call @ tap.js:16
subscribe @ Observable.js:26
Tl @ innerSubscribe.js:67
_innerSub @ switchMap.js:43
_next @ switchMap.js:33
next @ Subscriber.js:50
observe @ Notification.js:20
dispatch @ delay.js:34
_execute @ AsyncAction.js:50
execute @ AsyncAction.js:39
flush @ AsyncScheduler.js:33
w.<computed> @ zone.js:1705
invokeTask @ zone.js:356
onInvokeTask @ core.mjs:7018
invokeTask @ zone.js:356
runTask @ zone.js:157
invokeTask @ zone.js:433
E.useG.invoke @ zone.js:422
g.args.<computed> @ zone.js:1687
setInterval
c @ zone.js:1689
scheduleTask @ zone.js:346
onScheduleTask @ zone.js:257
scheduleTask @ zone.js:342
scheduleTask @ zone.js:196
scheduleMacroTask @ zone.js:218
Yn @ zone.js:647
(anonymous) @ zone.js:1727
n.<computed> @ zone.js:929
requestAsyncId @ AsyncAction.js:25
schedule @ AsyncAction.js:21
schedule @ Scheduler.ts:67
schedule @ AsyncScheduler.js:19
_schedule @ delay.js:47
scheduleNotification @ delay.js:61
_next @ delay.js:65
next @ Subscriber.js:50
notifyNext @ mergeMap.js:67
_next @ innerSubscribe.js:10
next @ Subscriber.js:50
notifyNext @ switchMap.js:67
_next @ innerSubscribe.js:10
next @ Subscriber.js:50
(anonymous) @ subscribeToArray.js:3
_trySubscribe @ Observable.js:42
subscribe @ Observable.js:28
Tl @ innerSubscribe.js:67
_innerSub @ switchMap.js:43
_next @ switchMap.js:33
next @ Subscriber.js:50
_next @ distinctUntilChanged.js:51
next @ Subscriber.js:50
_next @ map.js:34
next @ Subscriber.js:50
next @ Subject.js:42
next @ BehaviorSubject.js:28
countUp @ global-progress-bar.service.ts:60
(anonymous) @ sync-provider.service.ts:141
o @ main-7WUP32YU.js:1
invoke @ zone.js:331
onInvoke @ core.mjs:7029
invoke @ zone.js:331
run @ zone.js:111
(anonymous) @ zone.js:2366
invokeTask @ zone.js:356
onInvokeTask @ core.mjs:7018
invokeTask @ zone.js:356
runTask @ zone.js:157
Z @ zone.js:527
invokeTask @ zone.js:436
_ @ zone.js:1054
x @ zone.js:1084
K @ zone.js:1115
Show 99 more frames
Show less
sync-provider.service.ts:150 __error during sync__
sync-provider.service.ts:151 Error: Remote archive rev does not match the one in remote main file
    at e.<anonymous> (sync-provider.service.ts:702:15)
    at Generator.next (<anonymous>)
    at o (main-7WUP32YU.js:1:1557)
    at a.invoke (zone.js:331:158)
    at Object.onInvoke (core.mjs:7029:25)
    at a.invoke (zone.js:331:46)
    at b.run (zone.js:111:37)
    at zone.js:2366:30
    at a.invokeTask (zone.js:356:171)
    at Object.onInvokeTask (core.mjs:7018:25)
(anonymous) @ sync-provider.service.ts:151
s @ main-7WUP32YU.js:1
invoke @ zone.js:331
onInvoke @ core.mjs:7029
invoke @ zone.js:331
run @ zone.js:111
(anonymous) @ zone.js:2366
invokeTask @ zone.js:356
onInvokeTask @ core.mjs:7018
invokeTask @ zone.js:356
runTask @ zone.js:157
Z @ zone.js:527
invokeTask @ zone.js:436
E.useG.invoke @ zone.js:422
g.args.<computed> @ zone.js:1687
setTimeout
c @ zone.js:1689
scheduleTask @ zone.js:346
onScheduleTask @ zone.js:257
scheduleTask @ zone.js:342
scheduleTask @ zone.js:196
scheduleMacroTask @ zone.js:218
Yn @ zone.js:647
(anonymous) @ zone.js:1727
n.<computed> @ zone.js:929
_waitForAnimationToComplete @ dialog.mjs:209
_startExitAnimation @ dialog.mjs:169
close @ dialog.mjs:388
close @ dialog-incomplete-sync.component.ts:49
(anonymous) @ dialog-incomplete-sync.component.html:43
Wte @ core.mjs:25806
n @ core.mjs:25838
(anonymous) @ platform-browser.mjs:679
invokeTask @ zone.js:356
onInvokeTask @ core.mjs:7018
invokeTask @ zone.js:356
runTask @ zone.js:157
invokeTask @ zone.js:433
_ @ zone.js:1054
x @ zone.js:1084
K @ zone.js:1115
Show 36 more frames
Show less

(the "dataInStr" line is shortened, it just continues on with chinese letters...)

@codeling codeling added the bug label Oct 31, 2024
Copy link

Thank you very much for opening up this issue! I am currently a bit overwhelmed by the many requests that arrive each week, so please forgive me, if I fail to respond personally. I am still very likely to at least skim read your request and I'll probably try to fix all (real) bugs if possible and I will likely review every single PR being made (please, give me a heads up if you intent to do so) and I will try to work on popular requests (please upvote via thumbs up on the original issue) whenever possible, but trying to respond to every single issue over the last years has been kind of draining and I need to adjust my approach for this project to remain fun for me and to make any progress with actually coding new stuff. Thanks for your understanding!

@codeling
Copy link
Contributor Author

codeling commented Oct 31, 2024

I got it to work (for now, kind of):
What helped was to export the data from the "good" machine" and import it into the "problematic" machine.

Is this maybe the recommended way to set up a new machine anyway?

The bug I describe however also existed before - those two machines did synchronize successfully at some point, and only started showing the sync issue at some point in the past (probably with some update to superproductivity on one or the other side).

So me starting over on the one machine was just a way to try and get the sync to work again.

Having to do this export-import stuff for setting up feels a bit strange. For me the most logical way would have been (as I tried) to just point the sync to the same settings and fetch the remote state for a fresh install, but this does not seem not to work (though I suppose it should?)!

Just speculating, but from my experiences in this issue, it seems like for the sync to work, two superproductivity instances need a somewhat nebulous "common understanding" of their state; if the two machines somehow loose this "common understanding", then the sync stops working; therefore they need to be "re-acquainted" (by exporting the state of one, and importing it into the other).

Hopefully it was just some temporary glitch, and it works from here on out...

I'll try and set up superproductivity sync on a completely fresh machine and see what happens there!

@MonsterDruide1
Copy link

I was also running into this issue when setting up super-productivity on multiple devices. Similarly, my workaround was exporting/importing the data to get sync properly set up, after this initial weirdness, syncing worked just fine. I also noticed that I was able to get one of my devices properly synced up by pressing the Force Upload button, which apparently did not change anything on the uplodaded state - I guess that my previous attempts of setting up that device resulted in the proper data already being available/downloaded, so it uploaded the same data over the "save" already stored on WebDAV.

@codeling
Copy link
Contributor Author

codeling commented Nov 4, 2024

After working a while on the previously "problematic" machine (after the "fix" via the export/import workaround described above, and while superproductivity was not running on the previously "good" machine); I now get the "Last sync was incomplete" message on the previously "good" machine. On the first start of superproductivity there, I just got the "Last sync was incomplete" message; I exited the dialog to backup the local data; then disabling and re-enabling sync lead to the "conflict" dialog (where I chose "keep remote"), followed by the "last sync was incomplete" dialog...

I then tried the "force upload" button now as proposed by @MonsterDruide1 above, and indeed it seems like the data produced by the remote machine seems to stay available.

So the sync actually seems to work properly, only the "last sync was incomplete" dialog is shown erroneously?

@johannesjo
Copy link
Owner

johannesjo commented Nov 4, 2024

So the sync actually seems to work properly, only the "last sync was incomplete" dialog is shown erroneously?

The message is potentially misleading. Super Productivity expects the rev returned from the server to be consistent. In the screenshot you posted it is almost the same, but there is -gzip appended to it and so it assumes that something is wrong. Would be interesting to know why the -gzip string was attached in your case and what the underlying logic is. Maybe we can just ignore it, if it is present at the end? But maybe its also super specific and in only happens for your setup. Not sure :) Do you have an insights on that?

@codeling
Copy link
Contributor Author

codeling commented Nov 5, 2024

Super Productivity expects the rev returned from the server to be consistent. In the screenshot you posted it is almost the same, but there is -gzip appended to it and so it assumes that something is wrong. Would be interesting to know why the -gzip string was attached in your case and what the underlying logic is.

How is this rev string determined on the server / in superproductivity (it's the etag header, right)? I don't really have an idea where such a -gzip suffix could come from, specifically for my server configuration - it's just a normal nextcloud setup I think?
Pure speculation - maybe transfer compression could have something to do with it?

EDIT:
When checking via console (Ctrl+Shift+I), I do see 5 requests for MAIN.json per sync (HEAD GET HEAD(cached) PUT HEAD); some of them have the gzip suffix (2,3) in the Etag header, some don't (1,4,5). Requests 1-3 share the same etag "prefix", same as requests 4-5; all of them have an "Oc-Etag" which is always the "Etag" without any potential "-gzip" suffix (but this oc-etag seems to be something owncloud/nextcloud specific).

But yeah from everything I've read, apache appends the -gzip to the etag whenever it is transport-compressing; see e.g. here; not sure whether this is something standardized... seems I could configure apache to not append the -gzip suffix: stackoverflow / documentation. Not sure about the side-effects of that, though ;).

@johannesjo
Copy link
Owner

johannesjo commented Nov 5, 2024

Thanks for digging into this! I think in this case its safe, to just remove the string from the etag value whenever it is present from Super Productvity's side of things. I'll provide a fix soon!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants