make use of "sync tokens"

[rhodey]

Make use of sync tokens as specified by RFC6578 if they are supported by the WebDAV server. This should improve battery and network performance significantly.

[rhodey]

Right now I've got a mental blocks on implementing this feature, allow me to spend some time explaining the what and why of this feature so that I may be able to reach some conclusion...

WebDAV collections (eg calendars and address books) all have something called a CTag which is used to represent state of the entire collection. WebDAV collections contain resources (eg events and contacts) which each have an ETag. Whenever a resource within a collection is created, changed, or deleted the resources ETag changes but the CTag of its parent collection is also changed.

When the CTag of a collection changes Flock understands this means we need to figure out which resources have been created, changed, or deleted and mirror those changes locally. As of now Flock does this by requesting a list of ETags from the remote WebDAV collection and checking this list against the ETags within its local collection. This works, however if you have 5000 contacts the response from the server will contain 5000 ETags, their associated resource URI, and all the extra bloated bullshit that XML is loathed for, this results in an embarrassingly large HTTP response.

Sync Tokens basically allow us to request a list which only contains resource URIs that have changed since we last checked. The sync token acts just like the CTag except the server associates each sync token with a history of resource changes. So syncing remote changes would look like...

1. ask the server "show me everything that has changed since ".
2. use the response to mirror all remote changes locally.
3. save the new sync token of the remote collection.

Next time we need to sync this collection Flock will use the sync token saved in step 3 for the query in step 1, in here lies my mental block. When saving a sync token locally we're essentially saying "I am positive that this local collection is synced perfectly with that remote collection", we have to say this because by using a new sync token we are agreeing to forget about anything that happened with an old sync token. The reality is that there are cases in which the local collection does not sync perfectly with the remote collection and what to do with sync tokens in this case is not immediately obvious.

When pulling resources down from a remote server there are two things we really have to worry about:

1. InvalidMacExceptions - problem with decryption
2. InvalidComponentExceptions - problem with parsing a vcard or ical from decrypted blob.

Ideally we would never encounter these exceptions (and in my experience I have not) however in reality we definitely will encounter these and without an in-app debug log reporting mechanism we can't know how frequently they show up and thus cannot understand how to best resolve them. My thinking at this time is that if there is a remote resource which produces an InvalidMacException or InvalidComponentException the only real option is to delete it from the remote collection. Flock's behavior at this time is to simply log the exception and move on to the next resource.

If Flock implements sync tokens without modifying any of its other sync behavior then download of resources which cause InvalidMacException or InvalidComponentExceptions will only be attempted once. After they fail once we'll never see them again even though they will still remain on the remote server.

So, after the above rant I've reached the conclusion that we should not add support for sync tokens until:

1. in-app debug log reporting is supported.
2. we do more than just log InvalidMacExceptions and InvalidComponentExceptions

thoughts and advice greatly appreciated :)

[gusennan]

Know I'm reviving a dead thread here; but for future readers, going through the code it looks like with libpastelog the first requirement is met.