[DOCS] Document time sync issues as a common issue (or even a requirement)

[elasticmachine]

Original comment by @ppf2:

This is one of the action items identified at the call we scheduled with the Marvel team for LINK REDACTED .

The root cause turns out to be that the client browser used to view the Marvel UI is on a laptop that does not have the time server synced automatically (so that it was off by seconds). As a result, the most current indexing rate in the charts, tables, and everywhere were always showing 0.

It will be difficult to handle this in the product, so we decided at the call that it will probably be a good idea to start documenting this as one of the common issues, or a requirement to make sure that all servers (Marvel nodes, Kibana server) use the same NTP endpoints, and the Marvel client browser machine to also sync with a time server, etc..

[elasticmachine]

Original comment by @tsullivan:

I have seen a similar issue to this to a developer when running a cluster in a virtual machine, and the host machine clock was ahead. In that case, the Home page that lists the clusters actually said there was no Marvel Data indexed at all. The customer may have had the same thing happen, but I didn't see them click on the Home page during the session.

I assume that problem is easily reproducible and should be fixed.

[elasticmachine]

Original comment by @pickypg:

One simple thing that we should probably look at doing is to send the now time from the client to Marvel, and compare the timestamps for significant differences. If we detect one (I would probably suggest flagging anything above a minute), then we should warn the user.

For internet connected Marvel instances, we could possibly go an extra step and fetch the atomic time via a web service call.

Finally, we may want to look at trying to detect this from the Marvel Agent's broadcasts (maybe on first connect it can send some info like that, but this is probably the most complex).

[jrodewig]

[docs issue triage]