Transformation Gateway

Content Transformation Gateway

Program Name: chrysalis

The purpose of the transformation gateway is to take a primitive object supplied in an opensim/OMV internal format and convert it on the fly to a format that can be read by the web viewer.

Geometry

The input format will be a flatbuffer serialized version of the current protobuf primitive snapshot, but containing only the fields necessary to display the primitive (including physics fields for movement)

The output format will be a google flatbuffer object that mimics the structure of babylon.js's babylon file format for mesh geometry. This will be passed back to the arbiter and forwarded directly to the web viewer.

Textures

Textures can be retrieved via REST and are either looked up from a local texture cache or retrieved from the asset server on the fly and converted to a usable format to the web viewer. WebP format would be ideal here, but it has limited browser support (Currently chrome and opera only) Therefore, it may make sense to return PNG images for textures with transparency and JPG at a predetermined quality level for everything else to save on bandwidth.

Where possible we should directly pipe textures from the disk/cache to the client so that we can avoid garbage collection delays from buffering the image binary into memory.

Program

The gateway needs to be written in c# and as such it will be structured as an externally callable application.

Input/Output

The flatbuffer format is binary, and therefore should be carried on a protocol that supports binary data directly. As inventing protocols should be avoided where not firmly advantageous, we'll use an async HTTP server (likely system.net.httplistener) and REST semantics to deliver transformations.

Initial insight says the transforms will be mostly CPU bound. There is some I/O to pull assets, and these reads can be made async. Therefore an async HTTP server backed by a limited threadpool (configured based on CPUs available to the process) should maximize throughput.

Misc notes

The Transformation Gateway could likely be a bottleneck for the arbiter. Therefore we will make it stateless and scalable perhaps a single arbiter utilizing multiple gateways.