Data sync with video, including sensors & GeoJSON

[rjksmith]

Background

Mobile video devices often collect additional data, e.g. drones can be used as sensor platforms and dashcams record vehicle telemetry such as speed and acceleration.

Allowing the option to embed JSON data within cues would permit synchronisation of arbitrary data with location and video without compromising interoperability, in a similar fashion to the GPX <extension> element.

Proposal

The following design is proposed.

A WebVMT data sync command which consists of:

• A data sync type.
  The DataCue type of the associated metadata, proposed in the WICG DataCue Explainer, e.g. org.geojson.
• Data sync values.
  An array of data values, i.e. the raw data bytes, also proposed in the WICG DataCue Explainer.
• Optionally, a data sync id.
  An identifier to associate data samples over time, e.g. with a sensor or for interpolation.
• Optionally, a data sync path.
  The path identifier of the path segments associated with the data, which identifies a WebVMT path, i.e. an object moving through the mapped space.

Example

NOTE Cat, top left, after 5 secs until 25 secs

00:00:05.000 —> 00:00:25.000
{“sync”: {“type”: “org.ogc.geoai.example”, “data”:
  {“animal”:”cat”, “frame-zone”:”top-left"}
}}

NOTE Dog, mid right, after 10 secs until 40 secs

00:00:10.000 —> 00:00:40.000
{“sync”: {“type”: “org.ogc.geoai.example”, “data”:
  {“animal”:”dog”, “frame-zone”:”middle-right"}
}}

[rjksmith]

Issue migrated from w3c/sdw#1141